2023-12-12T09:34:20.745 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-12T09:34:20.746 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} --owner mchangir --verbose --name mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi --archive /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219 -- /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/orig.config.yaml 2023-12-12T09:34:20.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-12T09:34:20.885 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219 branch: wip-mchangir-testing-20231205.064351 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} email: mchangir@redhat.com first_in_suite: false job_id: '7488219' kernel: client: branch: testing flavor: default kdb: true sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-20231205.064351 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-20231205.064351 sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 owner: mchangir priority: 1000 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5473 sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 sleep_before_teardown: 0 subset: 600/1024 suite: fs suite_branch: wip-mchangir-testing-20231205.064351 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEU3ThwljGX0CbUGQkQeI+yI9VjYeXL1STRtECiXaiXc61gFHpsM3Tixc+RkxyInCiWcLjawjrMcsJUQgPtJVZU= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6xr8M/tal4s4/lDEJJ6DdIoRSlvP3w48DiCcXXkVGWpUf2hlO+7UPcu9si7kAnZ4/agxdjzwt+2abI7ruRLJg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_strays teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-12_08:46:52 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-12T09:34:20.885 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa; will attempt to use it 2023-12-12T09:34:20.886 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks 2023-12-12T09:34:20.886 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-12T09:34:20.887 INFO:teuthology.task.internal:Checking packages... 2023-12-12T09:34:20.912 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9' 2023-12-12T09:34:20.912 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-12T09:34:20.912 INFO:teuthology.packaging:ref: None 2023-12-12T09:34:20.912 INFO:teuthology.packaging:tag: None 2023-12-12T09:34:20.912 INFO:teuthology.packaging:branch: wip-mchangir-testing-20231205.064351 2023-12-12T09:34:20.912 INFO:teuthology.packaging:sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:34:20.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-20231205.064351 2023-12-12T09:34:21.133 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-96.gf1fbf3c5 2023-12-12T09:34:21.135 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-12T09:34:21.141 INFO:teuthology.task.internal:no buildpackages task found 2023-12-12T09:34:21.141 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-12T09:34:21.190 INFO:teuthology.task.internal:Saving configuration 2023-12-12T09:34:21.207 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-12T09:34:21.215 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-12T09:34:21.241 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:28:48.430886', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEU3ThwljGX0CbUGQkQeI+yI9VjYeXL1STRtECiXaiXc61gFHpsM3Tixc+RkxyInCiWcLjawjrMcsJUQgPtJVZU='} 2023-12-12T09:34:21.266 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:28:48.432884', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6xr8M/tal4s4/lDEJJ6DdIoRSlvP3w48DiCcXXkVGWpUf2hlO+7UPcu9si7kAnZ4/agxdjzwt+2abI7ruRLJg='} 2023-12-12T09:34:21.266 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-12T09:34:21.272 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-12T09:34:21.273 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-12T09:34:21.273 INFO:teuthology.run_tasks:Running task console_log... 2023-12-12T09:34:21.348 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff2f2016e50>, signals=[15]) 2023-12-12T09:34:21.349 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-12T09:34:21.355 INFO:teuthology.task.internal:Opening connections... 2023-12-12T09:34:21.355 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:34:21.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:34:21.442 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:34:21.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:34:21.529 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-12T09:34:21.536 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2023-12-12T09:34:21.553 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2023-12-12T09:34:21.553 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2023-12-12T09:34:21.610 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2023-12-12T09:34:21.610 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="9" 2023-12-12T09:34:21.610 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="9" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el9" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:LOGO="fedora-logo-icon" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-12T09:34:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-12T09:34:21.612 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-12-12T09:34:21.633 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2023-12-12T09:34:21.651 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2023-12-12T09:34:21.651 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2023-12-12T09:34:21.707 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2023-12-12T09:34:21.707 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2023-12-12T09:34:21.707 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2023-12-12T09:34:21.707 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2023-12-12T09:34:21.707 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-12T09:34:21.708 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-12T09:34:21.708 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2023-12-12T09:34:21.733 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-12T09:34:21.742 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-12T09:34:21.747 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-12T09:34:21.747 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2023-12-12T09:34:21.750 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2023-12-12T09:34:21.765 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-12T09:34:21.771 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-12T09:34:21.771 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2023-12-12T09:34:21.807 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2023-12-12T09:34:21.904 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-12T09:34:21.966 INFO:teuthology.run_tasks:Running task kernel... 2023-12-12T09:34:21.980 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'} 2023-12-12T09:34:21.981 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-12-12T09:34:21.981 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-12-12T09:34:21.981 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-12T09:34:21.981 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-12T09:34:21.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:34:21.998 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-12-12T09:34:22.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:34:22.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-12-12T09:34:22.213 DEBUG:teuthology.task.kernel:sha1 for client.0 is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-12T09:34:22.213 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-12T09:34:22.214 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-12-12T09:34:22.231 INFO:teuthology.orchestra.run.smithi072.stdout:5.14.0-361.el9.x86_64 2023-12-12T09:34:22.231 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-12T09:34:22.231 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-12T09:34:22.231 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum upgrade -y linux-firmware 2023-12-12T09:34:22.276 DEBUG:teuthology.task.kernel:sha1 for client.1 is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-12T09:34:22.276 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-12T09:34:22.276 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2023-12-12T09:34:22.293 INFO:teuthology.orchestra.run.smithi113.stdout:5.14.0-361.el9.x86_64 2023-12-12T09:34:22.294 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-12T09:34:22.294 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-12T09:34:22.294 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum upgrade -y linux-firmware 2023-12-12T09:34:24.344 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - AppStream 15 MB/s | 6.4 MB 00:00 2023-12-12T09:34:25.270 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2023-12-12T09:34:25.963 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2023-12-12T09:34:26.612 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2023-12-12T09:34:26.771 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 7.9 MB 00:00 2023-12-12T09:34:27.423 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 7.9 MB 00:00 2023-12-12T09:34:29.038 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 15 MB/s | 18 MB 00:01 2023-12-12T09:34:32.325 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 18 MB 00:03 2023-12-12T09:34:33.022 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 5.8 MB 00:00 2023-12-12T09:34:34.672 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 15 kB 00:00 2023-12-12T09:34:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 20 MB 00:02 2023-12-12T09:34:38.768 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 5.8 MB 00:03 2023-12-12T09:34:40.936 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 15 kB 00:01 2023-12-12T09:34:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2023-12-12T09:34:42.091 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 19 MB/s | 20 MB 00:01 2023-12-12T09:34:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2023-12-12T09:34:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2023-12-12T09:34:43.841 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2023-12-12T09:34:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2023-12-12T09:34:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2023-12-12T09:34:43.841 INFO:teuthology.orchestra.run.smithi113.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-12-12T09:34:43.841 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:34:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2023-12-12T09:34:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2023-12-12T09:34:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 1 Package 2023-12-12T09:34:43.842 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:34:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 328 M 2023-12-12T09:34:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2023-12-12T09:34:46.766 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2023-12-12T09:34:47.757 INFO:teuthology.orchestra.run.smithi113.stdout:linux-firmware-20231030-141.el9.noarch.rpm 84 MB/s | 328 MB 00:03 2023-12-12T09:34:47.758 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:34:47.759 INFO:teuthology.orchestra.run.smithi113.stdout:Total 84 MB/s | 328 MB 00:03 2023-12-12T09:34:47.759 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2023-12-12T09:34:47.845 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2023-12-12T09:34:47.846 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2023-12-12T09:34:49.007 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-12-12T09:34:49.021 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-12-12T09:34:49.022 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-12-12T09:34:49.022 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-12-12T09:34:49.022 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2023-12-12T09:34:49.022 INFO:teuthology.orchestra.run.smithi072.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-12-12T09:34:49.022 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:34:49.022 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-12-12T09:34:49.023 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-12-12T09:34:49.023 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 1 Package 2023-12-12T09:34:49.023 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:34:49.023 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 328 M 2023-12-12T09:34:49.023 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-12-12T09:34:49.998 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2023-12-12T09:34:49.998 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2023-12-12T09:34:54.941 INFO:teuthology.orchestra.run.smithi072.stdout:linux-firmware-20231030-141.el9.noarch.rpm 55 MB/s | 328 MB 00:05 2023-12-12T09:34:54.942 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:34:54.942 INFO:teuthology.orchestra.run.smithi072.stdout:Total 55 MB/s | 328 MB 00:05 2023-12-12T09:34:54.942 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-12-12T09:34:55.045 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-12-12T09:34:55.045 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-12-12T09:34:55.645 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2023-12-12T09:34:56.090 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-12T09:34:56.179 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-12T09:34:57.176 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-12-12T09:34:57.176 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-12-12T09:34:57.349 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-12-12T09:34:57.349 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-12T09:34:57.702 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-12T09:34:57.702 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:34:57.702 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2023-12-12T09:34:57.702 INFO:teuthology.orchestra.run.smithi113.stdout: linux-firmware-20231030-141.el9.noarch 2023-12-12T09:34:57.702 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:34:57.703 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2023-12-12T09:34:57.989 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on client.1... 2023-12-12T09:34:57.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-12T09:34:58.214 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/ 2023-12-12T09:34:58.214 DEBUG:teuthology.orchestra.run.smithi113:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/ --input-file=- 2023-12-12T09:34:58.217 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi113.front.sepia.ceph.com to complete... 2023-12-12T09:35:04.591 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-12-12T09:35:04.901 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-12T09:35:05.010 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-12T09:35:06.576 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-12-12T09:35:06.576 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-12T09:35:06.998 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-12T09:35:06.999 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:35:06.999 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2023-12-12T09:35:06.999 INFO:teuthology.orchestra.run.smithi072.stdout: linux-firmware-20231030-141.el9.noarch 2023-12-12T09:35:06.999 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:35:06.999 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-12-12T09:35:07.375 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on client.0... 2023-12-12T09:35:07.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-12T09:35:07.519 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/ 2023-12-12T09:35:07.519 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/ --input-file=- 2023-12-12T09:35:07.522 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi072.front.sepia.ceph.com to complete... 2023-12-12T09:35:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12 09:35:29 URL:https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [321162729/321162729] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-12T09:35:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:FINISHED --2023-12-12 09:35:29-- 2023-12-12T09:35:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:Total wall clock time: 22s 2023-12-12T09:35:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:Downloaded: 1 files, 306M in 22s (14.2 MB/s) 2023-12-12T09:35:29.454 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on client.0... 2023-12-12T09:35:29.455 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-12T09:35:30.134 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12 09:35:30 URL:https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [321162729/321162729] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-12T09:35:30.134 INFO:teuthology.orchestra.run.smithi113.stderr:FINISHED --2023-12-12 09:35:30-- 2023-12-12T09:35:30.134 INFO:teuthology.orchestra.run.smithi113.stderr:Total wall clock time: 32s 2023-12-12T09:35:30.134 INFO:teuthology.orchestra.run.smithi113.stderr:Downloaded: 1 files, 306M in 31s (9.73 MB/s) 2023-12-12T09:35:30.137 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on client.1... 2023-12-12T09:35:30.137 DEBUG:teuthology.orchestra.run.smithi113:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-12T09:35:31.299 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying... ######################################## 2023-12-12T09:35:31.371 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing... ######################################## 2023-12-12T09:35:31.450 INFO:teuthology.orchestra.run.smithi072.stdout:Updating / installing... 2023-12-12T09:35:31.992 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying... ######################################## 2023-12-12T09:35:32.131 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing... ######################################## 2023-12-12T09:35:32.210 INFO:teuthology.orchestra.run.smithi113.stdout:Updating / installing... 2023-12-12T09:35:38.513 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-12T09:35:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-12T09:35:52.886 INFO:teuthology.orchestra.run.smithi113.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-12T09:36:07.511 INFO:teuthology.orchestra.run.smithi072.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-12T09:36:14.153 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi113.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-12T09:36:14.153 DEBUG:teuthology.orchestra.run.smithi113:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-12T09:36:14.251 INFO:teuthology.orchestra.run.smithi113.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.251 INFO:teuthology.orchestra.run.smithi113.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-12T09:36:14.252 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-12T09:36:14.253 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-12T09:36:14.254 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-12T09:36:14.254 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-12T09:36:14.254 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-12T09:36:14.254 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-12T09:36:14.254 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-12T09:36:14.254 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-12T09:36:14.254 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-12T09:36:14.255 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-12T09:36:14.256 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-12T09:36:14.257 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-12T09:36:14.258 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-12T09:36:14.259 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-12T09:36:14.260 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-12T09:36:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-12T09:36:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-12T09:36:14.263 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-12T09:36:14.264 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-12T09:36:14.265 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-12T09:36:14.266 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-12T09:36:14.267 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-12T09:36:14.268 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-12T09:36:14.269 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-12T09:36:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-12T09:36:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-12T09:36:14.272 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-12T09:36:14.273 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-12T09:36:14.274 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-12T09:36:14.275 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-12T09:36:14.276 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-12T09:36:14.277 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-12T09:36:14.278 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-12T09:36:14.279 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-12T09:36:14.280 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-12T09:36:14.281 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-12T09:36:14.282 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-12T09:36:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-12T09:36:14.284 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-12T09:36:14.285 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-12T09:36:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-12T09:36:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-12T09:36:14.288 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-12T09:36:14.289 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-12T09:36:14.290 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-12T09:36:14.291 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-12T09:36:14.292 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-12T09:36:14.293 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-12T09:36:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-12T09:36:14.295 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-12T09:36:14.296 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-12T09:36:14.297 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-12T09:36:14.298 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-12T09:36:14.299 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-12T09:36:14.300 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-12T09:36:14.300 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-12T09:36:14.300 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-12T09:36:14.300 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-12T09:36:14.300 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-12T09:36:14.300 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-12T09:36:14.300 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-12T09:36:14.301 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-12T09:36:14.302 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-12T09:36:14.303 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-12T09:36:14.304 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-12T09:36:14.305 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-12T09:36:14.305 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-12T09:36:14.305 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-12T09:36:14.305 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-12T09:36:14.305 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-12T09:36:14.305 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-12T09:36:14.305 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-12T09:36:14.306 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-12T09:36:14.307 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-12T09:36:14.308 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-12T09:36:14.309 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-12T09:36:14.310 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-12T09:36:14.311 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-12T09:36:14.312 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-12T09:36:14.313 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-12T09:36:14.314 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-12T09:36:14.315 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-12T09:36:14.316 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-12T09:36:14.317 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-12T09:36:14.318 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-12T09:36:14.319 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-12T09:36:14.320 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-12T09:36:14.321 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-12T09:36:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-12T09:36:14.323 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-12T09:36:14.324 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-12T09:36:14.325 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-12T09:36:14.326 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-12T09:36:14.327 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-12T09:36:14.328 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-12T09:36:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-12T09:36:14.330 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-12T09:36:14.331 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-12T09:36:14.332 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-12T09:36:14.333 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-12T09:36:14.334 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-12T09:36:14.335 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-12T09:36:14.336 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-12T09:36:14.337 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-12T09:36:14.338 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-12T09:36:14.339 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-12T09:36:14.340 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-12T09:36:14.341 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-12T09:36:14.342 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-12T09:36:14.343 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-12T09:36:14.344 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-12T09:36:14.345 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-12T09:36:14.346 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-12T09:36:14.347 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-12T09:36:14.348 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-12T09:36:14.349 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-12T09:36:14.350 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-12T09:36:14.351 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-12T09:36:14.352 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-12T09:36:14.353 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-12T09:36:14.354 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-12T09:36:14.355 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-12T09:36:14.356 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-12T09:36:14.357 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-12T09:36:14.358 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-12T09:36:14.359 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-12T09:36:14.360 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-12T09:36:14.361 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-12T09:36:14.362 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-12T09:36:14.363 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-12T09:36:14.364 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-12T09:36:14.365 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-12T09:36:14.366 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-12T09:36:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-12T09:36:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-12T09:36:14.369 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-12T09:36:14.370 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-12T09:36:14.371 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-12T09:36:14.372 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-12T09:36:14.373 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-12T09:36:14.374 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-12T09:36:14.375 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-12T09:36:14.376 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-12T09:36:14.377 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-12T09:36:14.378 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-12T09:36:14.379 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-12T09:36:14.380 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-12T09:36:14.381 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-12T09:36:14.382 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.alias.bin 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-12T09:36:14.383 INFO:teuthology.orchestra.run.smithi113.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-12T09:36:14.384 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.384 DEBUG:teuthology.orchestra.run.smithi113:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-12T09:36:14.407 INFO:teuthology.orchestra.run.smithi113.stdout:postinstall scriptlet (using /bin/sh): 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:fi 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:fi 2023-12-12T09:36:14.408 INFO:teuthology.orchestra.run.smithi113.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-12T09:36:14.409 INFO:teuthology.orchestra.run.smithi113.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-12T09:36:14.409 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.409 INFO:teuthology.orchestra.run.smithi113.stdout:fi 2023-12-12T09:36:14.410 DEBUG:teuthology.orchestra.run.smithi113:> sudo rpm -qi grub2-tools 2023-12-12T09:36:14.485 INFO:teuthology.orchestra.run.smithi113.stdout:Name : grub2-tools 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Epoch : 1 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Version : 2.06 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Release : 68.el9 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Architecture: x86_64 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Group : Unspecified 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Size : 8264254 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:License : GPLv3+ 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Packager : builder@centos.org 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:Vendor : CentOS 2023-12-12T09:36:14.486 INFO:teuthology.orchestra.run.smithi113.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout:Summary : Support tools for GRUB. 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout:Description : 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout:hardware devices. 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:36:14.487 INFO:teuthology.orchestra.run.smithi113.stdout:This subpackage provides tools for support of all platforms. 2023-12-12T09:36:14.488 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-12T09:36:14.488 INFO:teuthology.task.kernel:Updating grub on smithi113 to boot 6.6.0-gcd68252d7b4d 2023-12-12T09:36:14.488 DEBUG:teuthology.orchestra.run.smithi113:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-12T09:36:15.092 INFO:teuthology.orchestra.run.smithi113.stderr:Generating grub configuration file ... 2023-12-12T09:36:16.486 INFO:teuthology.orchestra.run.smithi113.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-12T09:36:16.523 INFO:teuthology.orchestra.run.smithi113.stderr:done 2023-12-12T09:36:16.526 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2023-12-12T09:36:16.541 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.dBdYFkzb5C 2023-12-12T09:36:16.541 DEBUG:teuthology.orchestra.run.smithi113:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dBdYFkzb5C 2023-12-12T09:36:16.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.dBdYFkzb5C 2023-12-12T09:36:16.749 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.dBdYFkzb5C is 6KB 2023-12-12T09:36:16.798 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.dBdYFkzb5C 2023-12-12T09:36:16.812 DEBUG:teuthology.orchestra.run.smithi113:> sudo /bin/ls /boot/loader/entries || true 2023-12-12T09:36:16.881 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-12T09:36:16.881 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-12T09:36:16.881 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-12T09:36:16.881 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-6.6.0-gcd68252d7b4d.conf 2023-12-12T09:36:16.883 DEBUG:teuthology.orchestra.run.smithi113:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-gcd68252d7b4d 2023-12-12T09:36:17.029 DEBUG:teuthology.orchestra.run.smithi113:> sudo shutdown -r now 2023-12-12T09:36:30.070 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi072.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-12T09:36:30.070 DEBUG:teuthology.orchestra.run.smithi072:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-12T09:36:30.158 INFO:teuthology.orchestra.run.smithi072.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.158 INFO:teuthology.orchestra.run.smithi072.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.158 INFO:teuthology.orchestra.run.smithi072.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-12T09:36:30.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-12T09:36:30.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-12T09:36:30.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-12T09:36:30.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-12T09:36:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-12T09:36:30.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-12T09:36:30.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-12T09:36:30.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-12T09:36:30.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-12T09:36:30.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-12T09:36:30.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-12T09:36:30.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-12T09:36:30.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-12T09:36:30.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-12T09:36:30.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-12T09:36:30.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-12T09:36:30.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-12T09:36:30.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-12T09:36:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-12T09:36:30.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-12T09:36:30.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-12T09:36:30.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-12T09:36:30.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-12T09:36:30.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-12T09:36:30.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-12T09:36:30.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-12T09:36:30.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-12T09:36:30.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-12T09:36:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-12T09:36:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-12T09:36:30.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-12T09:36:30.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-12T09:36:30.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-12T09:36:30.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-12T09:36:30.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-12T09:36:30.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-12T09:36:30.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-12T09:36:30.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-12T09:36:30.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-12T09:36:30.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-12T09:36:30.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-12T09:36:30.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-12T09:36:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-12T09:36:30.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-12T09:36:30.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-12T09:36:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-12T09:36:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-12T09:36:30.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-12T09:36:30.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-12T09:36:30.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-12T09:36:30.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-12T09:36:30.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-12T09:36:30.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-12T09:36:30.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-12T09:36:30.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-12T09:36:30.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-12T09:36:30.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-12T09:36:30.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-12T09:36:30.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-12T09:36:30.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-12T09:36:30.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-12T09:36:30.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-12T09:36:30.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-12T09:36:30.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-12T09:36:30.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-12T09:36:30.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-12T09:36:30.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-12T09:36:30.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-12T09:36:30.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-12T09:36:30.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-12T09:36:30.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-12T09:36:30.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-12T09:36:30.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-12T09:36:30.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-12T09:36:30.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-12T09:36:30.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-12T09:36:30.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-12T09:36:30.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-12T09:36:30.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-12T09:36:30.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-12T09:36:30.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-12T09:36:30.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-12T09:36:30.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-12T09:36:30.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-12T09:36:30.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-12T09:36:30.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-12T09:36:30.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-12T09:36:30.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-12T09:36:30.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-12T09:36:30.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-12T09:36:30.246 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-12T09:36:30.247 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-12T09:36:30.248 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-12T09:36:30.249 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-12T09:36:30.250 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-12T09:36:30.251 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-12T09:36:30.252 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-12T09:36:30.253 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-12T09:36:30.254 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-12T09:36:30.255 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-12T09:36:30.256 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-12T09:36:30.257 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-12T09:36:30.258 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-12T09:36:30.259 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-12T09:36:30.260 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-12T09:36:30.261 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-12T09:36:30.262 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-12T09:36:30.263 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-12T09:36:30.264 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-12T09:36:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-12T09:36:30.266 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-12T09:36:30.267 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-12T09:36:30.268 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-12T09:36:30.269 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-12T09:36:30.270 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-12T09:36:30.271 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-12T09:36:30.272 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-12T09:36:30.273 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-12T09:36:30.274 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-12T09:36:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-12T09:36:30.276 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-12T09:36:30.277 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-12T09:36:30.278 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-12T09:36:30.279 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-12T09:36:30.280 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-12T09:36:30.281 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-12T09:36:30.282 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-12T09:36:30.283 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-12T09:36:30.284 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.alias.bin 2023-12-12T09:36:30.285 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-12T09:36:30.286 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-12T09:36:30.286 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-12T09:36:30.286 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-12T09:36:30.286 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-12T09:36:30.286 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-12T09:36:30.286 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-12T09:36:30.286 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-12T09:36:30.286 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-12T09:36:30.286 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.287 DEBUG:teuthology.orchestra.run.smithi072:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-12T09:36:30.309 INFO:teuthology.orchestra.run.smithi072.stdout:postinstall scriptlet (using /bin/sh): 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2023-12-12T09:36:30.310 INFO:teuthology.orchestra.run.smithi072.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-12T09:36:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-12T09:36:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.311 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2023-12-12T09:36:30.312 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -qi grub2-tools 2023-12-12T09:36:30.386 INFO:teuthology.orchestra.run.smithi072.stdout:Name : grub2-tools 2023-12-12T09:36:30.386 INFO:teuthology.orchestra.run.smithi072.stdout:Epoch : 1 2023-12-12T09:36:30.386 INFO:teuthology.orchestra.run.smithi072.stdout:Version : 2.06 2023-12-12T09:36:30.386 INFO:teuthology.orchestra.run.smithi072.stdout:Release : 68.el9 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Architecture: x86_64 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Group : Unspecified 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Size : 8264254 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:License : GPLv3+ 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Packager : builder@centos.org 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Vendor : CentOS 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Summary : Support tools for GRUB. 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout:Description : 2023-12-12T09:36:30.387 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:36:30.388 INFO:teuthology.orchestra.run.smithi072.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-12T09:36:30.388 INFO:teuthology.orchestra.run.smithi072.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-12T09:36:30.388 INFO:teuthology.orchestra.run.smithi072.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-12T09:36:30.388 INFO:teuthology.orchestra.run.smithi072.stdout:hardware devices. 2023-12-12T09:36:30.388 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:36:30.388 INFO:teuthology.orchestra.run.smithi072.stdout:This subpackage provides tools for support of all platforms. 2023-12-12T09:36:30.389 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-12T09:36:30.389 INFO:teuthology.task.kernel:Updating grub on smithi072 to boot 6.6.0-gcd68252d7b4d 2023-12-12T09:36:30.389 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-12T09:36:31.072 INFO:teuthology.orchestra.run.smithi072.stderr:Generating grub configuration file ... 2023-12-12T09:36:32.650 INFO:teuthology.orchestra.run.smithi072.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-12T09:36:32.675 INFO:teuthology.orchestra.run.smithi072.stderr:done 2023-12-12T09:36:32.678 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2023-12-12T09:36:32.701 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.OfpwU08o4P 2023-12-12T09:36:32.702 DEBUG:teuthology.orchestra.run.smithi072:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OfpwU08o4P 2023-12-12T09:36:32.770 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.OfpwU08o4P 2023-12-12T09:36:32.914 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.OfpwU08o4P is 6KB 2023-12-12T09:36:32.967 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.OfpwU08o4P 2023-12-12T09:36:32.981 DEBUG:teuthology.orchestra.run.smithi072:> sudo /bin/ls /boot/loader/entries || true 2023-12-12T09:36:33.048 INFO:teuthology.orchestra.run.smithi072.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-12T09:36:33.049 INFO:teuthology.orchestra.run.smithi072.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-12T09:36:33.049 INFO:teuthology.orchestra.run.smithi072.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-12T09:36:33.049 INFO:teuthology.orchestra.run.smithi072.stdout:15949e3c88704300882482eddfd7baae-6.6.0-gcd68252d7b4d.conf 2023-12-12T09:36:33.050 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-gcd68252d7b4d 2023-12-12T09:36:33.214 DEBUG:teuthology.orchestra.run.smithi072:> sudo shutdown -r now 2023-12-12T09:36:47.042 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-12-12T09:36:47.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-12T09:36:47.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:37:03.230 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-12-12T09:37:03.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-12-12T09:37:03.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:37:21.506 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-12T09:37:30.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-12T09:37:30.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:37:33.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-12T09:37:45.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-12T09:37:45.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:38:03.250 DEBUG:teuthology.orchestra.remote:timed out 2023-12-12T09:38:04.002 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-12T09:38:12.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-12-12T09:38:12.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:38:19.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-12T09:38:19.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:38:19.481 DEBUG:teuthology.orchestra.run.smithi113:> true 2023-12-12T09:38:20.209 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-12T09:38:20.209 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-gcd68252d7b4d"... 2023-12-12T09:38:20.209 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2023-12-12T09:38:20.258 INFO:teuthology.orchestra.run.smithi113.stdout:6.6.0-gcd68252d7b4d 2023-12-12T09:38:20.258 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-12T09:38:20.258 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-12T09:38:20.259 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-12T09:38:21.260 DEBUG:teuthology.parallel:result is None 2023-12-12T09:38:28.016 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-12-12T09:38:28.832 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-12-12T09:38:28.832 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-gcd68252d7b4d"... 2023-12-12T09:38:28.833 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-12-12T09:38:28.877 INFO:teuthology.orchestra.run.smithi072.stdout:6.6.0-gcd68252d7b4d 2023-12-12T09:38:28.878 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-12T09:38:28.878 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-12T09:38:28.878 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-12T09:38:29.881 DEBUG:teuthology.parallel:result is None 2023-12-12T09:38:29.881 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-12T09:38:29.890 INFO:teuthology.task.internal:Creating test directory... 2023-12-12T09:38:29.890 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-12T09:38:29.896 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-12T09:38:29.944 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-12T09:38:29.952 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-12T09:38:29.962 INFO:teuthology.task.internal:Creating archive directory... 2023-12-12T09:38:29.963 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-12T09:38:29.968 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-12T09:38:30.055 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-12T09:38:30.063 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-12T09:38:30.063 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-12T09:38:30.071 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-12T09:38:30.202 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:38:30.251 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:38:30.342 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:38:30.437 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:38:30.441 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-12T09:38:30.449 INFO:teuthology.task.internal:Configuring sudo... 2023-12-12T09:38:30.449 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-12T09:38:30.455 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-12T09:38:30.530 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-12T09:38:30.539 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-12T09:38:30.539 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-12T09:38:30.544 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-12T09:38:30.587 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:38:30.695 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:38:30.807 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:38:30.885 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:38:30.962 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:38:30.962 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-12T09:38:31.042 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:38:31.116 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:38:31.197 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:38:31.263 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:38:31.353 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:38:31.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-12T09:38:31.425 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2023-12-12T09:38:31.431 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2023-12-12T09:38:31.537 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T09:38:31.566 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T09:38:31.868 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-12T09:38:31.875 INFO:teuthology.task.internal:Starting timer... 2023-12-12T09:38:31.876 INFO:teuthology.run_tasks:Running task pcp... 2023-12-12T09:38:31.887 INFO:teuthology.run_tasks:Running task selinux... 2023-12-12T09:38:31.895 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2023-12-12T09:38:32.026 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2023-12-12T09:38:32.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2023-12-12T09:38:32.150 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2023-12-12T09:38:32.154 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-12T09:38:32.154 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2023-12-12T09:38:32.210 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2023-12-12T09:38:32.210 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2023-12-12T09:38:32.267 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2023-12-12T09:38:32.267 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive'} 2023-12-12T09:38:32.268 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T09:38:32.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:38:32.344 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T09:38:32.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:38:32.426 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-12T09:38:32.426 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2023-12-12T09:38:32.506 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2023-12-12T09:38:32.635 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-12T09:38:32.645 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-12T09:38:32.645 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-12T09:38:32.661 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-12T09:38:32.663 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi072.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-12T09:48:57.745 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2023-12-12T09:48:57.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-12-12T09:48:57.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:48:57.883 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-12-12T09:48:58.039 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-12-12T09:48:58.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-12T09:48:58.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:48:58.173 DEBUG:teuthology.orchestra.run.smithi113:> true 2023-12-12T09:48:58.323 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-12T09:48:58.324 INFO:teuthology.run_tasks:Running task clock... 2023-12-12T09:48:58.338 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-12T09:48:58.339 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-12T09:48:58.339 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T09:48:58.349 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-12T09:48:58.349 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T09:48:58.445 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-12T09:48:58.452 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-12T09:48:58.496 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-12T09:48:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-12T09:48:58.587 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2023-12-12T09:48:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2023-12-12T09:48:58.630 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2023-12-12T09:48:58.640 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2023-12-12T09:48:58.680 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-12T09:48:58.690 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-12T09:48:58.731 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-12T09:48:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-12T09:48:58.862 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2023-12-12T09:48:58.868 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2023-12-12T09:48:58.871 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T09:48:58.871 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2023-12-12T09:48:58.871 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:48:58.871 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:48:58.871 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:48:58.871 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:48:58.875 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T09:48:58.876 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2023-12-12T09:48:58.876 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:48:58.876 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:48:58.876 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:48:58.876 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:48:58.877 INFO:teuthology.run_tasks:Running task install... 2023-12-12T09:48:58.886 DEBUG:teuthology.task.install:project ceph 2023-12-12T09:48:58.886 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'}} 2023-12-12T09:48:58.886 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'} 2023-12-12T09:48:58.886 INFO:teuthology.task.install:Using flavor: default 2023-12-12T09:48:58.893 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-12T09:48:58.893 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-12T09:48:58.893 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'tag': None, 'wait_for_package': False} 2023-12-12T09:48:58.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:48:58.896 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'tag': None, 'wait_for_package': False} 2023-12-12T09:48:58.897 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:48:59.065 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/9/flavors/default/ 2023-12-12T09:48:59.066 INFO:teuthology.task.install.rpm:Package version is 19.0.0-96.gf1fbf3c5 2023-12-12T09:48:59.203 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/9/flavors/default/ 2023-12-12T09:48:59.203 INFO:teuthology.task.install.rpm:Package version is 19.0.0-96.gf1fbf3c5 2023-12-12T09:48:59.294 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-12T09:48:59.295 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:48:59.295 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-12T09:48:59.307 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-12T09:48:59.307 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:48:59.307 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-12T09:48:59.390 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-12T09:48:59.390 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-12T09:48:59.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-12T09:48:59.410 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-12T09:48:59.530 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-12T09:48:59.552 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-12T09:48:59.663 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-12T09:48:59.688 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-12T09:48:59.754 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2023-12-12T09:48:59.757 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2023-12-12T09:48:59.776 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2023-12-12T09:48:59.780 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2023-12-12T09:49:00.113 INFO:teuthology.orchestra.run.smithi072.stdout:73 files removed 2023-12-12T09:49:00.141 INFO:teuthology.orchestra.run.smithi113.stdout:73 files removed 2023-12-12T09:49:00.151 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-12T09:49:00.180 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-12T09:49:01.175 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 153 kB/s | 78 kB 00:00 2023-12-12T09:49:01.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 136 kB/s | 78 kB 00:00 2023-12-12T09:49:01.726 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 21 kB/s | 9.5 kB 00:00 2023-12-12T09:49:01.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 22 kB/s | 9.5 kB 00:00 2023-12-12T09:49:02.294 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2023-12-12T09:49:02.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2023-12-12T09:49:02.942 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-9 - AppStream 11 MB/s | 6.4 MB 00:00 2023-12-12T09:49:02.959 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-12-12T09:49:04.501 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2023-12-12T09:49:04.626 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2023-12-12T09:49:06.522 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 7.9 MB 00:01 2023-12-12T09:49:06.700 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 7.9 MB 00:01 2023-12-12T09:49:08.582 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 21 MB/s | 18 MB 00:00 2023-12-12T09:49:09.401 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 12 MB/s | 18 MB 00:01 2023-12-12T09:49:13.799 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 5.8 MB 00:01 2023-12-12T09:49:14.430 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 5.8 MB 00:01 2023-12-12T09:49:15.722 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 15 kB 00:00 2023-12-12T09:49:16.123 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2023-12-12T09:49:17.240 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2023-12-12T09:49:17.874 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2023-12-12T09:49:22.781 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2023-12-12T09:49:23.396 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2023-12-12T09:49:25.419 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-12T09:49:25.421 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-12T09:49:25.441 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-12T09:49:25.442 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-12T09:49:25.443 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-12T09:49:25.445 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-12T09:49:25.447 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-12T09:49:25.454 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-12T09:49:25.454 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-12T09:49:25.531 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2023-12-12T09:49:25.550 INFO:teuthology.orchestra.run.smithi113.stdout:========================================================================================== 2023-12-12T09:49:25.550 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2023-12-12T09:49:25.550 INFO:teuthology.orchestra.run.smithi113.stdout:========================================================================================== 2023-12-12T09:49:25.550 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2023-12-12T09:49:25.550 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-12T09:49:25.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 6.5 k 2023-12-12T09:49:25.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 5.2 M 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 860 k 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 141 k 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 1.5 M 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 133 k 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 1.8 M 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 7.4 M 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 50 k 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 8.6 M 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 41 M 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 262 k 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 803 k 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 217 k 2023-12-12T09:49:25.551 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 18 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 31 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 718 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 126 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-12T09:49:25.552 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 161 k 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 320 k 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 297 k 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 99 k 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 85 k 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 3.0 M 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 170 k 2023-12-12T09:49:25.553 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-12T09:49:25.554 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 3.3 M 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 3.0 M 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 19 M 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 28 k 2023-12-12T09:49:25.555 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 2.1 M 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 243 k 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 4.4 M 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 18 M 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 15 k 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 24 k 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-12T09:49:25.556 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 163 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 476 k 2023-12-12T09:49:25.557 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 4.8 M 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-12T09:49:25.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 45 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 129 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-12T09:49:25.559 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-12T09:49:25.560 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-12T09:49:25.561 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-12T09:49:25.562 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout:========================================================================================== 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout:Install 134 Packages 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 15 Packages 2023-12-12T09:49:25.563 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:49:25.564 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 204 M 2023-12-12T09:49:25.564 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2023-12-12T09:49:26.337 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-12T09:49:26.339 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-12T09:49:26.361 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-12T09:49:26.362 INFO:teuthology.orchestra.run.smithi072.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-12T09:49:26.363 INFO:teuthology.orchestra.run.smithi072.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-12T09:49:26.365 INFO:teuthology.orchestra.run.smithi072.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-12T09:49:26.367 INFO:teuthology.orchestra.run.smithi072.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-12T09:49:26.371 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-12T09:49:26.373 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-12T09:49:26.416 INFO:teuthology.orchestra.run.smithi113.stdout:(1/149): ceph-19.0.0-96.gf1fbf3c5.el9.x86_64.rp 19 kB/s | 6.5 kB 00:00 2023-12-12T09:49:26.458 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-12-12T09:49:26.484 INFO:teuthology.orchestra.run.smithi072.stdout:========================================================================================== 2023-12-12T09:49:26.484 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout:========================================================================================== 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 6.5 k 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 5.2 M 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 860 k 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 141 k 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 1.5 M 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 133 k 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 1.8 M 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 7.4 M 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 50 k 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 8.6 M 2023-12-12T09:49:26.485 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 41 M 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 262 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 803 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 217 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 18 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 31 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 718 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 126 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-12T09:49:26.486 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 161 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 320 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 297 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 99 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 85 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 3.0 M 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 170 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-12T09:49:26.487 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 3.3 M 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 3.0 M 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-12T09:49:26.488 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 19 M 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 28 k 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 2.1 M 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 243 k 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 4.4 M 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 18 M 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:19.0.0-96.gf1fbf3c5.el9 ceph-noarch 15 k 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 24 k 2023-12-12T09:49:26.489 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 163 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-12T09:49:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 476 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 4.8 M 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-12T09:49:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 45 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el9 ceph 129 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-12T09:49:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-12T09:49:26.493 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-12T09:49:26.494 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-12T09:49:26.495 INFO:teuthology.orchestra.run.smithi072.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout:========================================================================================== 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout:Install 134 Packages 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 15 Packages 2023-12-12T09:49:26.496 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:49:26.497 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 204 M 2023-12-12T09:49:26.497 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-12-12T09:49:27.041 INFO:teuthology.orchestra.run.smithi113.stdout:(2/149): ceph-fuse-19.0.0-96.gf1fbf3c5.el9.x86_ 1.3 MB/s | 860 kB 00:00 2023-12-12T09:49:27.191 INFO:teuthology.orchestra.run.smithi113.stdout:(3/149): ceph-immutable-object-cache-19.0.0-96. 947 kB/s | 141 kB 00:00 2023-12-12T09:49:27.307 INFO:teuthology.orchestra.run.smithi113.stdout:(4/149): ceph-base-19.0.0-96.gf1fbf3c5.el9.x86_ 4.2 MB/s | 5.2 MB 00:01 2023-12-12T09:49:27.432 INFO:teuthology.orchestra.run.smithi072.stdout:(1/149): ceph-19.0.0-96.gf1fbf3c5.el9.x86_64.rp 19 kB/s | 6.5 kB 00:00 2023-12-12T09:49:27.525 INFO:teuthology.orchestra.run.smithi113.stdout:(5/149): ceph-mgr-19.0.0-96.gf1fbf3c5.el9.x86_6 6.8 MB/s | 1.5 MB 00:00 2023-12-12T09:49:28.358 INFO:teuthology.orchestra.run.smithi113.stdout:(6/149): ceph-mds-19.0.0-96.gf1fbf3c5.el9.x86_6 1.8 MB/s | 2.1 MB 00:01 2023-12-12T09:49:28.416 INFO:teuthology.orchestra.run.smithi072.stdout:(2/149): ceph-fuse-19.0.0-96.gf1fbf3c5.el9.x86_ 875 kB/s | 860 kB 00:00 2023-12-12T09:49:28.566 INFO:teuthology.orchestra.run.smithi072.stdout:(3/149): ceph-immutable-object-cache-19.0.0-96. 942 kB/s | 141 kB 00:00 2023-12-12T09:49:28.725 INFO:teuthology.orchestra.run.smithi113.stdout:(7/149): ceph-common-19.0.0-96.gf1fbf3c5.el9.x8 7.1 MB/s | 19 MB 00:02 2023-12-12T09:49:28.834 INFO:teuthology.orchestra.run.smithi113.stdout:(8/149): ceph-mon-19.0.0-96.gf1fbf3c5.el9.x86_6 3.4 MB/s | 4.4 MB 00:01 2023-12-12T09:49:28.951 INFO:teuthology.orchestra.run.smithi113.stdout:(9/149): ceph-selinux-19.0.0-96.gf1fbf3c5.el9.x 208 kB/s | 24 kB 00:00 2023-12-12T09:49:29.225 INFO:teuthology.orchestra.run.smithi072.stdout:(4/149): ceph-base-19.0.0-96.gf1fbf3c5.el9.x86_ 2.4 MB/s | 5.2 MB 00:02 2023-12-12T09:49:29.659 INFO:teuthology.orchestra.run.smithi072.stdout:(5/149): ceph-mds-19.0.0-96.gf1fbf3c5.el9.x86_6 1.9 MB/s | 2.1 MB 00:01 2023-12-12T09:49:29.659 INFO:teuthology.orchestra.run.smithi113.stdout:(10/149): ceph-radosgw-19.0.0-96.gf1fbf3c5.el9. 9.2 MB/s | 8.6 MB 00:00 2023-12-12T09:49:29.751 INFO:teuthology.orchestra.run.smithi113.stdout:(11/149): cephfs-mirror-19.0.0-96.gf1fbf3c5.el9 2.3 MB/s | 217 kB 00:00 2023-12-12T09:49:29.785 INFO:teuthology.orchestra.run.smithi072.stdout:(6/149): ceph-mgr-19.0.0-96.gf1fbf3c5.el9.x86_6 2.6 MB/s | 1.5 MB 00:00 2023-12-12T09:49:29.843 INFO:teuthology.orchestra.run.smithi113.stdout:(12/149): libcephfs-devel-19.0.0-96.gf1fbf3c5.e 342 kB/s | 31 kB 00:00 2023-12-12T09:49:29.968 INFO:teuthology.orchestra.run.smithi113.stdout:(13/149): libcephfs2-19.0.0-96.gf1fbf3c5.el9.x8 5.7 MB/s | 718 kB 00:00 2023-12-12T09:49:30.060 INFO:teuthology.orchestra.run.smithi113.stdout:(14/149): libcephsqlite-19.0.0-96.gf1fbf3c5.el9 1.8 MB/s | 163 kB 00:00 2023-12-12T09:49:30.154 INFO:teuthology.orchestra.run.smithi113.stdout:(15/149): librados-devel-19.0.0-96.gf1fbf3c5.el 1.4 MB/s | 126 kB 00:00 2023-12-12T09:49:30.269 INFO:teuthology.orchestra.run.smithi113.stdout:(16/149): libradosstriper1-19.0.0-96.gf1fbf3c5. 4.0 MB/s | 476 kB 00:00 2023-12-12T09:49:31.003 INFO:teuthology.orchestra.run.smithi113.stdout:(17/149): librgw2-19.0.0-96.gf1fbf3c5.el9.x86_6 6.5 MB/s | 4.8 MB 00:00 2023-12-12T09:49:31.245 INFO:teuthology.orchestra.run.smithi113.stdout:(18/149): ceph-osd-19.0.0-96.gf1fbf3c5.el9.x86_ 6.1 MB/s | 18 MB 00:02 2023-12-12T09:49:31.286 INFO:teuthology.orchestra.run.smithi113.stdout:(19/149): python3-ceph-argparse-19.0.0-96.gf1fb 159 kB/s | 45 kB 00:00 2023-12-12T09:49:31.320 INFO:teuthology.orchestra.run.smithi113.stdout:(20/149): python3-ceph-common-19.0.0-96.gf1fbf3 1.7 MB/s | 129 kB 00:00 2023-12-12T09:49:31.370 INFO:teuthology.orchestra.run.smithi113.stdout:(21/149): python3-cephfs-19.0.0-96.gf1fbf3c5.el 1.9 MB/s | 161 kB 00:00 2023-12-12T09:49:31.404 INFO:teuthology.orchestra.run.smithi113.stdout:(22/149): python3-rados-19.0.0-96.gf1fbf3c5.el9 3.8 MB/s | 320 kB 00:00 2023-12-12T09:49:31.454 INFO:teuthology.orchestra.run.smithi113.stdout:(23/149): python3-rbd-19.0.0-96.gf1fbf3c5.el9.x 3.5 MB/s | 297 kB 00:00 2023-12-12T09:49:31.488 INFO:teuthology.orchestra.run.smithi113.stdout:(24/149): python3-rgw-19.0.0-96.gf1fbf3c5.el9.x 1.2 MB/s | 99 kB 00:00 2023-12-12T09:49:31.530 INFO:teuthology.orchestra.run.smithi113.stdout:(25/149): rbd-fuse-19.0.0-96.gf1fbf3c5.el9.x86_ 1.1 MB/s | 85 kB 00:00 2023-12-12T09:49:31.604 INFO:teuthology.orchestra.run.smithi072.stdout:(7/149): ceph-mon-19.0.0-96.gf1fbf3c5.el9.x86_6 2.3 MB/s | 4.4 MB 00:01 2023-12-12T09:49:31.614 INFO:teuthology.orchestra.run.smithi113.stdout:(26/149): rbd-nbd-19.0.0-96.gf1fbf3c5.el9.x86_6 2.0 MB/s | 170 kB 00:00 2023-12-12T09:49:31.730 INFO:teuthology.orchestra.run.smithi113.stdout:(27/149): ceph-grafana-dashboards-19.0.0-96.gf1 243 kB/s | 28 kB 00:00 2023-12-12T09:49:31.856 INFO:teuthology.orchestra.run.smithi113.stdout:(28/149): rbd-mirror-19.0.0-96.gf1fbf3c5.el9.x8 8.2 MB/s | 3.0 MB 00:00 2023-12-12T09:49:31.889 INFO:teuthology.orchestra.run.smithi113.stdout:(29/149): ceph-mgr-cephadm-19.0.0-96.gf1fbf3c5. 842 kB/s | 133 kB 00:00 2023-12-12T09:49:32.131 INFO:teuthology.orchestra.run.smithi113.stdout:(30/149): ceph-mgr-dashboard-19.0.0-96.gf1fbf3c 6.4 MB/s | 1.8 MB 00:00 2023-12-12T09:49:32.232 INFO:teuthology.orchestra.run.smithi113.stdout:(31/149): ceph-mgr-modules-core-19.0.0-96.gf1fb 2.4 MB/s | 243 kB 00:00 2023-12-12T09:49:32.332 INFO:teuthology.orchestra.run.smithi113.stdout:(32/149): ceph-mgr-rook-19.0.0-96.gf1fbf3c5.el9 506 kB/s | 50 kB 00:00 2023-12-12T09:49:32.424 INFO:teuthology.orchestra.run.smithi113.stdout:(33/149): ceph-prometheus-alerts-19.0.0-96.gf1f 166 kB/s | 15 kB 00:00 2023-12-12T09:49:32.549 INFO:teuthology.orchestra.run.smithi113.stdout:(34/149): ceph-volume-19.0.0-96.gf1fbf3c5.el9.n 2.1 MB/s | 262 kB 00:00 2023-12-12T09:49:32.724 INFO:teuthology.orchestra.run.smithi113.stdout:(35/149): cephadm-19.0.0-96.gf1fbf3c5.el9.noarc 4.5 MB/s | 803 kB 00:00 2023-12-12T09:49:32.860 INFO:teuthology.orchestra.run.smithi113.stdout:(36/149): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2023-12-12T09:49:32.883 INFO:teuthology.orchestra.run.smithi113.stdout:(37/149): cephfs-top-19.0.0-96.gf1fbf3c5.el9.no 115 kB/s | 18 kB 00:00 2023-12-12T09:49:32.962 INFO:teuthology.orchestra.run.smithi072.stdout:(8/149): ceph-common-19.0.0-96.gf1fbf3c5.el9.x8 3.2 MB/s | 19 MB 00:05 2023-12-12T09:49:33.038 INFO:teuthology.orchestra.run.smithi072.stdout:(9/149): ceph-selinux-19.0.0-96.gf1fbf3c5.el9.x 324 kB/s | 24 kB 00:00 2023-12-12T09:49:33.134 INFO:teuthology.orchestra.run.smithi113.stdout:(38/149): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2023-12-12T09:49:33.184 INFO:teuthology.orchestra.run.smithi113.stdout:(39/149): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 920 kB 00:00 2023-12-12T09:49:33.209 INFO:teuthology.orchestra.run.smithi113.stdout:(40/149): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2023-12-12T09:49:33.235 INFO:teuthology.orchestra.run.smithi113.stdout:(41/149): elfutils-libelf-devel-0.190-1.el9.x86 441 kB/s | 21 kB 00:00 2023-12-12T09:49:33.268 INFO:teuthology.orchestra.run.smithi113.stdout:(42/149): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2023-12-12T09:49:33.294 INFO:teuthology.orchestra.run.smithi113.stdout:(43/149): flexiblas-3.0.4-8.el9.x86_64.rpm 574 kB/s | 32 kB 00:00 2023-12-12T09:49:33.452 INFO:teuthology.orchestra.run.smithi113.stdout:(44/149): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2023-12-12T09:49:33.477 INFO:teuthology.orchestra.run.smithi113.stdout:(45/149): flexiblas-openblas-openmp-3.0.4-8.el9 102 kB/s | 17 kB 00:00 2023-12-12T09:49:33.519 INFO:teuthology.orchestra.run.smithi113.stdout:(46/149): libaio-devel-0.3.111-13.el9.x86_64.rp 250 kB/s | 10 kB 00:00 2023-12-12T09:49:33.544 INFO:teuthology.orchestra.run.smithi113.stdout:(47/149): jq-1.6-16.el9.x86_64.rpm 2.8 MB/s | 186 kB 00:00 2023-12-12T09:49:33.555 INFO:teuthology.orchestra.run.smithi072.stdout:(10/149): ceph-osd-19.0.0-96.gf1fbf3c5.el9.x86_ 4.7 MB/s | 18 MB 00:03 2023-12-12T09:49:33.570 INFO:teuthology.orchestra.run.smithi113.stdout:(48/149): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2023-12-12T09:49:33.595 INFO:teuthology.orchestra.run.smithi113.stdout:(49/149): librabbitmq-0.11.0-7.el9.x86_64.rpm 921 kB/s | 44 kB 00:00 2023-12-12T09:49:33.637 INFO:teuthology.orchestra.run.smithi113.stdout:(50/149): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 661 kB 00:00 2023-12-12T09:49:33.671 INFO:teuthology.orchestra.run.smithi113.stdout:(51/149): libtool-2.4.6-45.el9.x86_64.rpm 8.2 MB/s | 578 kB 00:00 2023-12-12T09:49:33.680 INFO:teuthology.orchestra.run.smithi072.stdout:(11/149): cephfs-mirror-19.0.0-96.gf1fbf3c5.el9 1.7 MB/s | 217 kB 00:00 2023-12-12T09:49:33.688 INFO:teuthology.orchestra.run.smithi113.stdout:(52/149): libuuid-devel-2.37.4-15.el9.x86_64.rp 542 kB/s | 24 kB 00:00 2023-12-12T09:49:33.713 INFO:teuthology.orchestra.run.smithi113.stdout:(53/149): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2023-12-12T09:49:33.747 INFO:teuthology.orchestra.run.smithi113.stdout:(54/149): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-12-12T09:49:33.747 INFO:teuthology.orchestra.run.smithi072.stdout:(12/149): libcephfs-devel-19.0.0-96.gf1fbf3c5.e 472 kB/s | 31 kB 00:00 2023-12-12T09:49:33.780 INFO:teuthology.orchestra.run.smithi113.stdout:(55/149): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2023-12-12T09:49:33.814 INFO:teuthology.orchestra.run.smithi113.stdout:(56/149): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-12-12T09:49:33.839 INFO:teuthology.orchestra.run.smithi113.stdout:(57/149): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 34 kB 00:00 2023-12-12T09:49:33.915 INFO:teuthology.orchestra.run.smithi072.stdout:(13/149): libcephfs2-19.0.0-96.gf1fbf3c5.el9.x8 4.2 MB/s | 718 kB 00:00 2023-12-12T09:49:33.981 INFO:teuthology.orchestra.run.smithi113.stdout:(58/149): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2023-12-12T09:49:33.990 INFO:teuthology.orchestra.run.smithi072.stdout:(14/149): libcephsqlite-19.0.0-96.gf1fbf3c5.el9 2.2 MB/s | 163 kB 00:00 2023-12-12T09:49:34.031 INFO:teuthology.orchestra.run.smithi113.stdout:(59/149): postgresql-13.11-1.el9.x86_64.rpm 8.5 MB/s | 1.5 MB 00:00 2023-12-12T09:49:34.066 INFO:teuthology.orchestra.run.smithi072.stdout:(15/149): librados-devel-19.0.0-96.gf1fbf3c5.el 1.7 MB/s | 126 kB 00:00 2023-12-12T09:49:34.073 INFO:teuthology.orchestra.run.smithi113.stdout:(60/149): postgresql-contrib-13.11-1.el9.x86_64 8.7 MB/s | 811 kB 00:00 2023-12-12T09:49:34.099 INFO:teuthology.orchestra.run.smithi113.stdout:(61/149): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2023-12-12T09:49:34.257 INFO:teuthology.orchestra.run.smithi113.stdout:(62/149): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2023-12-12T09:49:34.342 INFO:teuthology.orchestra.run.smithi072.stdout:(16/149): ceph-radosgw-19.0.0-96.gf1fbf3c5.el9. 3.1 MB/s | 8.6 MB 00:02 2023-12-12T09:49:34.383 INFO:teuthology.orchestra.run.smithi072.stdout:(17/149): libradosstriper1-19.0.0-96.gf1fbf3c5. 1.5 MB/s | 476 kB 00:00 2023-12-12T09:49:34.456 INFO:teuthology.orchestra.run.smithi113.stdout:(63/149): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-12-12T09:49:34.459 INFO:teuthology.orchestra.run.smithi072.stdout:(18/149): python3-ceph-argparse-19.0.0-96.gf1fb 605 kB/s | 45 kB 00:00 2023-12-12T09:49:34.508 INFO:teuthology.orchestra.run.smithi113.stdout:(64/149): python3-jinja2-2.11.3-4.el9.noarch.rp 967 kB/s | 228 kB 00:00 2023-12-12T09:49:34.535 INFO:teuthology.orchestra.run.smithi072.stdout:(19/149): python3-ceph-common-19.0.0-96.gf1fbf3 1.7 MB/s | 129 kB 00:00 2023-12-12T09:49:34.550 INFO:teuthology.orchestra.run.smithi113.stdout:(65/149): python3-jsonpointer-2.0-4.el9.noarch. 447 kB/s | 18 kB 00:00 2023-12-12T09:49:34.575 INFO:teuthology.orchestra.run.smithi113.stdout:(66/149): python3-jsonpatch-1.21-16.el9.noarch. 370 kB/s | 24 kB 00:00 2023-12-12T09:49:34.610 INFO:teuthology.orchestra.run.smithi072.stdout:(20/149): python3-cephfs-19.0.0-96.gf1fbf3c5.el 2.1 MB/s | 161 kB 00:00 2023-12-12T09:49:34.617 INFO:teuthology.orchestra.run.smithi113.stdout:(67/149): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2023-12-12T09:49:34.651 INFO:teuthology.orchestra.run.smithi113.stdout:(68/149): python3-markupsafe-1.1.1-12.el9.x86_6 431 kB/s | 32 kB 00:00 2023-12-12T09:49:34.719 INFO:teuthology.orchestra.run.smithi072.stdout:(21/149): python3-rados-19.0.0-96.gf1fbf3c5.el9 2.9 MB/s | 320 kB 00:00 2023-12-12T09:49:34.785 INFO:teuthology.orchestra.run.smithi113.stdout:(69/149): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2023-12-12T09:49:34.805 INFO:teuthology.orchestra.run.smithi072.stdout:(22/149): python3-rbd-19.0.0-96.gf1fbf3c5.el9.x 3.5 MB/s | 297 kB 00:00 2023-12-12T09:49:34.809 INFO:teuthology.orchestra.run.smithi113.stdout:(70/149): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2023-12-12T09:49:34.843 INFO:teuthology.orchestra.run.smithi113.stdout:(71/149): python3-prettytable-0.7.2-27.el9.noar 1.3 MB/s | 41 kB 00:00 2023-12-12T09:49:34.869 INFO:teuthology.orchestra.run.smithi113.stdout:(72/149): python3-oauthlib-3.1.1-5.el9.noarch.r 3.0 MB/s | 174 kB 00:00 2023-12-12T09:49:34.880 INFO:teuthology.orchestra.run.smithi072.stdout:(23/149): python3-rgw-19.0.0-96.gf1fbf3c5.el9.x 1.3 MB/s | 99 kB 00:00 2023-12-12T09:49:34.894 INFO:teuthology.orchestra.run.smithi113.stdout:(73/149): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2023-12-12T09:49:34.919 INFO:teuthology.orchestra.run.smithi113.stdout:(74/149): python3-pyasn1-modules-0.4.8-6.el9.no 4.3 MB/s | 211 kB 00:00 2023-12-12T09:49:34.944 INFO:teuthology.orchestra.run.smithi113.stdout:(75/149): python3-pytz-2021.1-5.el9.noarch.rpm 994 kB/s | 47 kB 00:00 2023-12-12T09:49:34.947 INFO:teuthology.orchestra.run.smithi072.stdout:(24/149): rbd-fuse-19.0.0-96.gf1fbf3c5.el9.x86_ 1.3 MB/s | 85 kB 00:00 2023-12-12T09:49:34.970 INFO:teuthology.orchestra.run.smithi113.stdout:(76/149): python3-requests-oauthlib-1.3.0-12.el 875 kB/s | 43 kB 00:00 2023-12-12T09:49:35.245 INFO:teuthology.orchestra.run.smithi113.stdout:(77/149): python3-scipy-1.6.2-8.el9.x86_64.rpm 52 MB/s | 16 MB 00:00 2023-12-12T09:49:35.270 INFO:teuthology.orchestra.run.smithi113.stdout:(78/149): python3-toml-0.10.2-6.el9.noarch.rpm 135 kB/s | 39 kB 00:00 2023-12-12T09:49:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:(79/149): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2023-12-12T09:49:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:(80/149): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2023-12-12T09:49:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:(81/149): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 85 kB 00:00 2023-12-12T09:49:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:(82/149): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2023-12-12T09:49:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:(83/149): zlib-devel-1.2.11-40.el9.x86_64.rpm 769 kB/s | 44 kB 00:00 2023-12-12T09:49:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:(84/149): fuse-2.9.9-15.el9.x86_64.rpm 954 kB/s | 79 kB 00:00 2023-12-12T09:49:35.523 INFO:teuthology.orchestra.run.smithi072.stdout:(25/149): rbd-mirror-19.0.0-96.gf1fbf3c5.el9.x8 5.2 MB/s | 3.0 MB 00:00 2023-12-12T09:49:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:(85/149): libgfortran-11.4.1-2.3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2023-12-12T09:49:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:(86/149): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2023-12-12T09:49:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:(87/149): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 241 kB 00:00 2023-12-12T09:49:35.666 INFO:teuthology.orchestra.run.smithi072.stdout:(26/149): librgw2-19.0.0-96.gf1fbf3c5.el9.x86_6 3.6 MB/s | 4.8 MB 00:01 2023-12-12T09:49:35.699 INFO:teuthology.orchestra.run.smithi072.stdout:(27/149): rbd-nbd-19.0.0-96.gf1fbf3c5.el9.x86_6 971 kB/s | 170 kB 00:00 2023-12-12T09:49:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:(88/149): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.1 MB 00:00 2023-12-12T09:49:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:(89/149): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2023-12-12T09:49:35.733 INFO:teuthology.orchestra.run.smithi072.stdout:(28/149): ceph-grafana-dashboards-19.0.0-96.gf1 423 kB/s | 28 kB 00:00 2023-12-12T09:49:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:(90/149): python3-pycparser-2.20-6.el9.noarch.r 3.0 MB/s | 124 kB 00:00 2023-12-12T09:49:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:(91/149): python3-requests-2.25.1-7.el9.noarch. 2.3 MB/s | 113 kB 00:00 2023-12-12T09:49:35.775 INFO:teuthology.orchestra.run.smithi072.stdout:(29/149): ceph-mgr-cephadm-19.0.0-96.gf1fbf3c5. 1.7 MB/s | 133 kB 00:00 2023-12-12T09:49:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:(92/149): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-12-12T09:49:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:(93/149): python3-devel-3.9.17-2.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2023-12-12T09:49:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:(94/149): openssl-devel-3.0.7-25.el9.x86_64.rpm 12 MB/s | 4.1 MB 00:00 2023-12-12T09:49:36.176 INFO:teuthology.orchestra.run.smithi072.stdout:(30/149): ceph-mgr-dashboard-19.0.0-96.gf1fbf3c 4.0 MB/s | 1.8 MB 00:00 2023-12-12T09:49:36.302 INFO:teuthology.orchestra.run.smithi072.stdout:(31/149): ceph-mgr-modules-core-19.0.0-96.gf1fb 1.9 MB/s | 243 kB 00:00 2023-12-12T09:49:36.394 INFO:teuthology.orchestra.run.smithi072.stdout:(32/149): ceph-mgr-rook-19.0.0-96.gf1fbf3c5.el9 549 kB/s | 50 kB 00:00 2023-12-12T09:49:36.494 INFO:teuthology.orchestra.run.smithi072.stdout:(33/149): ceph-prometheus-alerts-19.0.0-96.gf1f 152 kB/s | 15 kB 00:00 2023-12-12T09:49:36.620 INFO:teuthology.orchestra.run.smithi072.stdout:(34/149): ceph-volume-19.0.0-96.gf1fbf3c5.el9.n 2.1 MB/s | 262 kB 00:00 2023-12-12T09:49:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:(95/149): gperftools-libs-2.9.1-2.el9.x86_64.rp 458 kB/s | 309 kB 00:00 2023-12-12T09:49:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:(96/149): libarrow-doc-9.0.0-10.el9.noarch.rpm 196 kB/s | 26 kB 00:00 2023-12-12T09:49:36.847 INFO:teuthology.orchestra.run.smithi072.stdout:(35/149): cephadm-19.0.0-96.gf1fbf3c5.el9.noarc 3.5 MB/s | 803 kB 00:00 2023-12-12T09:49:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:(97/149): liboath-2.6.7-2.el9.x86_64.rpm 278 kB/s | 49 kB 00:00 2023-12-12T09:49:37.031 INFO:teuthology.orchestra.run.smithi072.stdout:(36/149): ceph-mgr-diskprediction-local-19.0.0- 5.9 MB/s | 7.4 MB 00:01 2023-12-12T09:49:37.057 INFO:teuthology.orchestra.run.smithi072.stdout:(37/149): cephfs-top-19.0.0-96.gf1fbf3c5.el9.no 87 kB/s | 18 kB 00:00 2023-12-12T09:49:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:(98/149): libarrow-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:00 2023-12-12T09:49:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:(99/149): libunwind-1.6.2-1.el9.x86_64.rpm 353 kB/s | 67 kB 00:00 2023-12-12T09:49:37.274 INFO:teuthology.orchestra.run.smithi072.stdout:(38/149): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2023-12-12T09:49:37.316 INFO:teuthology.orchestra.run.smithi072.stdout:(39/149): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 920 kB 00:00 2023-12-12T09:49:37.350 INFO:teuthology.orchestra.run.smithi072.stdout:(40/149): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2023-12-12T09:49:37.376 INFO:teuthology.orchestra.run.smithi072.stdout:(41/149): elfutils-libelf-devel-0.190-1.el9.x86 373 kB/s | 21 kB 00:00 2023-12-12T09:49:37.418 INFO:teuthology.orchestra.run.smithi072.stdout:(42/149): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2023-12-12T09:49:37.444 INFO:teuthology.orchestra.run.smithi072.stdout:(43/149): flexiblas-3.0.4-8.el9.x86_64.rpm 494 kB/s | 32 kB 00:00 2023-12-12T09:49:37.561 INFO:teuthology.orchestra.run.smithi072.stdout:(44/149): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2023-12-12T09:49:37.587 INFO:teuthology.orchestra.run.smithi072.stdout:(45/149): flexiblas-openblas-openmp-3.0.4-8.el9 134 kB/s | 17 kB 00:00 2023-12-12T09:49:37.637 INFO:teuthology.orchestra.run.smithi072.stdout:(46/149): jq-1.6-16.el9.x86_64.rpm 3.7 MB/s | 186 kB 00:00 2023-12-12T09:49:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:(100/149): ceph-test-19.0.0-96.gf1fbf3c5.el9.x8 4.7 MB/s | 41 MB 00:08 2023-12-12T09:49:37.662 INFO:teuthology.orchestra.run.smithi072.stdout:(47/149): libaio-devel-0.3.111-13.el9.x86_64.rp 135 kB/s | 10 kB 00:00 2023-12-12T09:49:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:(101/149): luarocks-3.9.2-1.el9.noarch.rpm 263 kB/s | 151 kB 00:00 2023-12-12T09:49:37.705 INFO:teuthology.orchestra.run.smithi072.stdout:(48/149): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2023-12-12T09:49:37.738 INFO:teuthology.orchestra.run.smithi072.stdout:(49/149): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-12-12T09:49:37.790 INFO:teuthology.orchestra.run.smithi072.stdout:(50/149): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2023-12-12T09:49:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:(102/149): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2023-12-12T09:49:37.843 INFO:teuthology.orchestra.run.smithi072.stdout:(51/149): libtool-2.4.6-45.el9.x86_64.rpm 5.9 MB/s | 578 kB 00:00 2023-12-12T09:49:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:(103/149): python3-bcrypt-3.2.2-1.el9.x86_64.rp 275 kB/s | 43 kB 00:00 2023-12-12T09:49:37.867 INFO:teuthology.orchestra.run.smithi072.stdout:(52/149): libuuid-devel-2.37.4-15.el9.x86_64.rp 349 kB/s | 24 kB 00:00 2023-12-12T09:49:37.893 INFO:teuthology.orchestra.run.smithi072.stdout:(53/149): libzstd-devel-1.5.1-2.el9.x86_64.rpm 934 kB/s | 46 kB 00:00 2023-12-12T09:49:37.936 INFO:teuthology.orchestra.run.smithi072.stdout:(54/149): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2023-12-12T09:49:37.969 INFO:teuthology.orchestra.run.smithi072.stdout:(55/149): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2023-12-12T09:49:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:(104/149): python3-cachetools-4.2.4-1.el9.noarc 215 kB/s | 32 kB 00:00 2023-12-12T09:49:38.012 INFO:teuthology.orchestra.run.smithi072.stdout:(56/149): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2023-12-12T09:49:38.037 INFO:teuthology.orchestra.run.smithi072.stdout:(57/149): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 34 kB 00:00 2023-12-12T09:49:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:(105/149): python3-certifi-2021.10.8-2.el9.noar 82 kB/s | 15 kB 00:00 2023-12-12T09:49:38.170 INFO:teuthology.orchestra.run.smithi113.stdout:(106/149): python3-cheroot-8.6.0-4.el9.noarch.r 861 kB/s | 172 kB 00:00 2023-12-12T09:49:38.204 INFO:teuthology.orchestra.run.smithi072.stdout:(58/149): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2023-12-12T09:49:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:(107/149): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2023-12-12T09:49:38.271 INFO:teuthology.orchestra.run.smithi072.stdout:(59/149): postgresql-13.11-1.el9.x86_64.rpm 6.9 MB/s | 1.5 MB 00:00 2023-12-12T09:49:38.330 INFO:teuthology.orchestra.run.smithi072.stdout:(60/149): postgresql-contrib-13.11-1.el9.x86_64 6.4 MB/s | 811 kB 00:00 2023-12-12T09:49:38.364 INFO:teuthology.orchestra.run.smithi072.stdout:(61/149): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2023-12-12T09:49:38.396 INFO:teuthology.orchestra.run.smithi113.stdout:(108/149): python3-asyncssh-2.13.2-3.el9.noarch 746 kB/s | 548 kB 00:00 2023-12-12T09:49:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:(109/149): python3-google-auth-2.23.4-1.el9.noa 764 kB/s | 197 kB 00:00 2023-12-12T09:49:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:(110/149): python3-jaraco-8.2.1-3.el9.noarch.rp 49 kB/s | 11 kB 00:00 2023-12-12T09:49:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:(111/149): python3-jaraco-classes-3.2.1-5.el9.n 112 kB/s | 18 kB 00:00 2023-12-12T09:49:38.564 INFO:teuthology.orchestra.run.smithi072.stdout:(62/149): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 5.8 MB 00:00 2023-12-12T09:49:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:(112/149): python3-jaraco-collections-3.0.0-8.e 121 kB/s | 23 kB 00:00 2023-12-12T09:49:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:(113/149): python3-jaraco-functools-3.5.0-2.el9 83 kB/s | 19 kB 00:00 2023-12-12T09:49:38.715 INFO:teuthology.orchestra.run.smithi072.stdout:(63/149): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2023-12-12T09:49:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:(114/149): python3-jaraco-text-3.2.0-6.el9.noar 102 kB/s | 20 kB 00:00 2023-12-12T09:49:38.750 INFO:teuthology.orchestra.run.smithi072.stdout:(64/149): python3-jinja2-2.11.3-4.el9.noarch.rp 1.2 MB/s | 228 kB 00:00 2023-12-12T09:49:38.776 INFO:teuthology.orchestra.run.smithi072.stdout:(65/149): python3-jsonpatch-1.21-16.el9.noarch. 422 kB/s | 24 kB 00:00 2023-12-12T09:49:38.801 INFO:teuthology.orchestra.run.smithi072.stdout:(66/149): python3-jsonpointer-2.0-4.el9.noarch. 363 kB/s | 18 kB 00:00 2023-12-12T09:49:38.827 INFO:teuthology.orchestra.run.smithi072.stdout:(67/149): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2023-12-12T09:49:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:(115/149): python3-jwt+crypto-2.4.0-1.el9.noarc 54 kB/s | 9.0 kB 00:00 2023-12-12T09:49:38.852 INFO:teuthology.orchestra.run.smithi072.stdout:(68/149): python3-markupsafe-1.1.1-12.el9.x86_6 647 kB/s | 32 kB 00:00 2023-12-12T09:49:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:(116/149): python3-jwt-2.4.0-1.el9.noarch.rpm 233 kB/s | 41 kB 00:00 2023-12-12T09:49:39.028 INFO:teuthology.orchestra.run.smithi072.stdout:(69/149): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2023-12-12T09:49:39.062 INFO:teuthology.orchestra.run.smithi072.stdout:(70/149): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-12-12T09:49:39.104 INFO:teuthology.orchestra.run.smithi072.stdout:(71/149): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2023-12-12T09:49:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:(117/149): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2023-12-12T09:49:39.163 INFO:teuthology.orchestra.run.smithi072.stdout:(72/149): python3-prettytable-0.7.2-27.el9.noar 415 kB/s | 41 kB 00:00 2023-12-12T09:49:39.205 INFO:teuthology.orchestra.run.smithi072.stdout:(73/149): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2023-12-12T09:49:39.239 INFO:teuthology.orchestra.run.smithi072.stdout:(74/149): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2023-12-12T09:49:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:(118/149): python3-more-itertools-8.12.0-2.el9. 215 kB/s | 79 kB 00:00 2023-12-12T09:49:39.274 INFO:teuthology.orchestra.run.smithi072.stdout:(75/149): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2023-12-12T09:49:39.298 INFO:teuthology.orchestra.run.smithi072.stdout:(76/149): python3-requests-oauthlib-1.3.0-12.el 743 kB/s | 43 kB 00:00 2023-12-12T09:49:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:(119/149): python3-logutils-0.3.5-21.el9.noarch 99 kB/s | 46 kB 00:00 2023-12-12T09:49:39.332 INFO:teuthology.orchestra.run.smithi072.stdout:(77/149): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 39 kB 00:00 2023-12-12T09:49:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:(120/149): python3-natsort-7.1.1-5.el9.noarch.r 238 kB/s | 58 kB 00:00 2023-12-12T09:49:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:(121/149): python3-portend-3.1.0-2.el9.noarch.r 104 kB/s | 16 kB 00:00 2023-12-12T09:49:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:(122/149): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2023-12-12T09:49:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:(123/149): python3-pyOpenSSL-21.0.0-1.el9.noarc 360 kB/s | 90 kB 00:00 2023-12-12T09:49:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:(124/149): python3-repoze-lru-0.7-16.el9.noarch 142 kB/s | 31 kB 00:00 2023-12-12T09:49:39.699 INFO:teuthology.orchestra.run.smithi072.stdout:(78/149): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2023-12-12T09:49:39.732 INFO:teuthology.orchestra.run.smithi072.stdout:(79/149): socat-1.7.4.1-5.el9.x86_64.rpm 777 kB/s | 300 kB 00:00 2023-12-12T09:49:39.742 INFO:teuthology.orchestra.run.smithi113.stdout:(125/149): python3-rsa-4.9-2.el9.noarch.rpm 418 kB/s | 59 kB 00:00 2023-12-12T09:49:39.767 INFO:teuthology.orchestra.run.smithi072.stdout:(80/149): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 2.6 MB/s | 85 kB 00:00 2023-12-12T09:49:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:(126/149): python3-routes-2.5.1-5.el9.noarch.rp 685 kB/s | 188 kB 00:00 2023-12-12T09:49:39.794 INFO:teuthology.orchestra.run.smithi072.stdout:(81/149): uuid-1.6.2-55.el9.x86_64.rpm 945 kB/s | 56 kB 00:00 2023-12-12T09:49:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:(127/149): python3-tempora-5.0.0-2.el9.noarch.r 252 kB/s | 36 kB 00:00 2023-12-12T09:49:39.828 INFO:teuthology.orchestra.run.smithi072.stdout:(82/149): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2023-12-12T09:49:39.861 INFO:teuthology.orchestra.run.smithi072.stdout:(83/149): zlib-devel-1.2.11-40.el9.x86_64.rpm 664 kB/s | 44 kB 00:00 2023-12-12T09:49:39.896 INFO:teuthology.orchestra.run.smithi072.stdout:(84/149): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2023-12-12T09:49:39.947 INFO:teuthology.orchestra.run.smithi072.stdout:(85/149): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.5 MB/s | 803 kB 00:00 2023-12-12T09:49:39.980 INFO:teuthology.orchestra.run.smithi072.stdout:(86/149): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2023-12-12T09:49:40.014 INFO:teuthology.orchestra.run.smithi072.stdout:(87/149): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2023-12-12T09:49:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:(128/149): python3-typing-extensions-4.4.0-2.el 180 kB/s | 51 kB 00:00 2023-12-12T09:49:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:(129/149): python3-webob-1.8.7-6.el9.noarch.rpm 787 kB/s | 230 kB 00:00 2023-12-12T09:49:40.077 INFO:teuthology.orchestra.run.smithi113.stdout:(130/149): python3-websocket-client-1.2.3-2.el9 347 kB/s | 90 kB 00:00 2023-12-12T09:49:40.081 INFO:teuthology.orchestra.run.smithi072.stdout:(88/149): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2023-12-12T09:49:40.115 INFO:teuthology.orchestra.run.smithi072.stdout:(89/149): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2023-12-12T09:49:40.183 INFO:teuthology.orchestra.run.smithi072.stdout:(90/149): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2023-12-12T09:49:40.217 INFO:teuthology.orchestra.run.smithi072.stdout:(91/149): python3-requests-2.25.1-7.el9.noarch. 1.1 MB/s | 113 kB 00:00 2023-12-12T09:49:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:(131/149): python3-zc-lockfile-2.0-10.el9.noarc 120 kB/s | 20 kB 00:00 2023-12-12T09:49:40.250 INFO:teuthology.orchestra.run.smithi072.stdout:(92/149): python3-urllib3-1.26.5-4.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2023-12-12T09:49:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:(132/149): python3-werkzeug-2.0.3-3.el9.1.noarc 1.5 MB/s | 427 kB 00:00 2023-12-12T09:49:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:(133/149): re2-20211101-3.el9.x86_64.rpm 590 kB/s | 192 kB 00:00 2023-12-12T09:49:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:(134/149): thrift-0.15.0-2.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2023-12-12T09:49:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:(135/149): libacl-devel-2.3.1-4.el9.x86_64.rpm 912 kB/s | 68 kB 00:00 2023-12-12T09:49:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:(136/149): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 69 kB 00:00 2023-12-12T09:49:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:(137/149): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-12-12T09:49:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:(138/149): elfutils-libelf-0.190-1.el9.x86_64.r 4.6 MB/s | 192 kB 00:00 2023-12-12T09:49:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:(139/149): elfutils-libs-0.190-1.el9.x86_64.rpm 6.1 MB/s | 255 kB 00:00 2023-12-12T09:49:40.909 INFO:teuthology.orchestra.run.smithi072.stdout:(93/149): python3-devel-3.9.17-2.el9.x86_64.rpm 373 kB/s | 245 kB 00:00 2023-12-12T09:49:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:(140/149): libacl-2.3.1-4.el9.x86_64.rpm 672 kB/s | 22 kB 00:00 2023-12-12T09:49:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:(141/149): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2023-12-12T09:49:41.078 INFO:teuthology.orchestra.run.smithi072.stdout:(94/149): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.8 MB/s | 309 kB 00:00 2023-12-12T09:49:41.239 INFO:teuthology.orchestra.run.smithi113.stdout:(142/149): librados2-19.0.0-96.gf1fbf3c5.el9.x8 3.5 MB/s | 3.3 MB 00:00 2023-12-12T09:49:41.362 INFO:teuthology.orchestra.run.smithi072.stdout:(95/149): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2023-12-12T09:49:41.413 INFO:teuthology.orchestra.run.smithi072.stdout:(96/149): libarrow-doc-9.0.0-10.el9.noarch.rpm 527 kB/s | 26 kB 00:00 2023-12-12T09:49:41.463 INFO:teuthology.orchestra.run.smithi072.stdout:(97/149): liboath-2.6.7-2.el9.x86_64.rpm 980 kB/s | 49 kB 00:00 2023-12-12T09:49:41.514 INFO:teuthology.orchestra.run.smithi072.stdout:(98/149): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2023-12-12T09:49:41.573 INFO:teuthology.orchestra.run.smithi072.stdout:(99/149): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2023-12-12T09:49:41.673 INFO:teuthology.orchestra.run.smithi072.stdout:(100/149): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2023-12-12T09:49:41.749 INFO:teuthology.orchestra.run.smithi072.stdout:(101/149): python3-asyncssh-2.13.2-3.el9.noarch 7.2 MB/s | 548 kB 00:00 2023-12-12T09:49:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:(102/149): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2023-12-12T09:49:41.834 INFO:teuthology.orchestra.run.smithi072.stdout:(103/149): python3-cachetools-4.2.4-1.el9.noarc 782 kB/s | 32 kB 00:00 2023-12-12T09:49:41.876 INFO:teuthology.orchestra.run.smithi072.stdout:(104/149): python3-certifi-2021.10.8-2.el9.noar 364 kB/s | 15 kB 00:00 2023-12-12T09:49:41.927 INFO:teuthology.orchestra.run.smithi072.stdout:(105/149): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2023-12-12T09:49:41.986 INFO:teuthology.orchestra.run.smithi072.stdout:(106/149): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2023-12-12T09:49:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:(143/149): librbd1-19.0.0-96.gf1fbf3c5.el9.x86_ 1.9 MB/s | 3.0 MB 00:01 2023-12-12T09:49:42.036 INFO:teuthology.orchestra.run.smithi072.stdout:(107/149): python3-google-auth-2.23.4-1.el9.noa 3.9 MB/s | 197 kB 00:00 2023-12-12T09:49:42.080 INFO:teuthology.orchestra.run.smithi072.stdout:(108/149): python3-jaraco-8.2.1-3.el9.noarch.rp 250 kB/s | 11 kB 00:00 2023-12-12T09:49:42.122 INFO:teuthology.orchestra.run.smithi072.stdout:(109/149): python3-jaraco-classes-3.2.1-5.el9.n 429 kB/s | 18 kB 00:00 2023-12-12T09:49:42.164 INFO:teuthology.orchestra.run.smithi072.stdout:(110/149): python3-jaraco-collections-3.0.0-8.e 562 kB/s | 23 kB 00:00 2023-12-12T09:49:42.241 INFO:teuthology.orchestra.run.smithi072.stdout:(111/149): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2023-12-12T09:49:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:(112/149): python3-jaraco-text-3.2.0-6.el9.noar 478 kB/s | 20 kB 00:00 2023-12-12T09:49:42.333 INFO:teuthology.orchestra.run.smithi072.stdout:(113/149): python3-jwt+crypto-2.4.0-1.el9.noarc 183 kB/s | 9.0 kB 00:00 2023-12-12T09:49:42.375 INFO:teuthology.orchestra.run.smithi072.stdout:(114/149): python3-jwt-2.4.0-1.el9.noarch.rpm 996 kB/s | 41 kB 00:00 2023-12-12T09:49:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:(144/149): NetworkManager-1.45.7-1.el9.x86_64.r 1.3 MB/s | 2.3 MB 00:01 2023-12-12T09:49:42.872 INFO:teuthology.orchestra.run.smithi113.stdout:(145/149): NetworkManager-tui-1.45.7-1.el9.x86_ 1.5 MB/s | 246 kB 00:00 2023-12-12T09:49:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:(146/149): NetworkManager-team-1.45.7-1.el9.x86 46 kB/s | 43 kB 00:00 2023-12-12T09:49:43.117 INFO:teuthology.orchestra.run.smithi072.stdout:(115/149): ceph-test-19.0.0-96.gf1fbf3c5.el9.x8 4.1 MB/s | 41 MB 00:10 2023-12-12T09:49:43.285 INFO:teuthology.orchestra.run.smithi072.stdout:(116/149): openssl-devel-3.0.7-25.el9.x86_64.rp 1.3 MB/s | 4.1 MB 00:03 2023-12-12T09:49:43.344 INFO:teuthology.orchestra.run.smithi072.stdout:(117/149): python3-kubernetes-26.1.0-2.el9.noar 1.1 MB/s | 1.0 MB 00:00 2023-12-12T09:49:43.369 INFO:teuthology.orchestra.run.smithi072.stdout:(118/149): python3-logutils-0.3.5-21.el9.noarch 185 kB/s | 46 kB 00:00 2023-12-12T09:49:43.411 INFO:teuthology.orchestra.run.smithi072.stdout:(119/149): python3-natsort-7.1.1-5.el9.noarch.r 867 kB/s | 58 kB 00:00 2023-12-12T09:49:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:(147/149): openssl-3.0.7-25.el9.x86_64.rpm 2.2 MB/s | 1.2 MB 00:00 2023-12-12T09:49:43.445 INFO:teuthology.orchestra.run.smithi072.stdout:(120/149): python3-more-itertools-8.12.0-2.el9. 495 kB/s | 79 kB 00:00 2023-12-12T09:49:43.479 INFO:teuthology.orchestra.run.smithi072.stdout:(121/149): python3-portend-3.1.0-2.el9.noarch.r 247 kB/s | 16 kB 00:00 2023-12-12T09:49:43.523 INFO:teuthology.orchestra.run.smithi072.stdout:(122/149): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2023-12-12T09:49:43.556 INFO:teuthology.orchestra.run.smithi072.stdout:(123/149): python3-pyOpenSSL-21.0.0-1.el9.noarc 815 kB/s | 90 kB 00:00 2023-12-12T09:49:43.582 INFO:teuthology.orchestra.run.smithi072.stdout:(124/149): python3-repoze-lru-0.7-16.el9.noarch 301 kB/s | 31 kB 00:00 2023-12-12T09:49:43.616 INFO:teuthology.orchestra.run.smithi072.stdout:(125/149): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2023-12-12T09:49:43.650 INFO:teuthology.orchestra.run.smithi072.stdout:(126/149): python3-tempora-5.0.0-2.el9.noarch.r 537 kB/s | 36 kB 00:00 2023-12-12T09:49:43.684 INFO:teuthology.orchestra.run.smithi072.stdout:(127/149): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2023-12-12T09:49:43.709 INFO:teuthology.orchestra.run.smithi072.stdout:(128/149): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2023-12-12T09:49:43.751 INFO:teuthology.orchestra.run.smithi072.stdout:(129/149): python3-websocket-client-1.2.3-2.el9 2.2 MB/s | 90 kB 00:00 2023-12-12T09:49:43.785 INFO:teuthology.orchestra.run.smithi072.stdout:(130/149): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2023-12-12T09:49:43.844 INFO:teuthology.orchestra.run.smithi072.stdout:(131/149): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2023-12-12T09:49:43.886 INFO:teuthology.orchestra.run.smithi072.stdout:(132/149): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2023-12-12T09:49:43.920 INFO:teuthology.orchestra.run.smithi072.stdout:(133/149): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-12-12T09:49:44.004 INFO:teuthology.orchestra.run.smithi072.stdout:(134/149): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-12-12T09:49:44.046 INFO:teuthology.orchestra.run.smithi072.stdout:(135/149): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-12-12T09:49:44.089 INFO:teuthology.orchestra.run.smithi072.stdout:(136/149): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2023-12-12T09:49:44.124 INFO:teuthology.orchestra.run.smithi072.stdout:(137/149): elfutils-debuginfod-client-0.190-1.e 1.0 MB/s | 36 kB 00:00 2023-12-12T09:49:44.166 INFO:teuthology.orchestra.run.smithi072.stdout:(138/149): elfutils-libelf-0.190-1.el9.x86_64.r 4.6 MB/s | 192 kB 00:00 2023-12-12T09:49:44.209 INFO:teuthology.orchestra.run.smithi072.stdout:(139/149): elfutils-libs-0.190-1.el9.x86_64.rpm 6.1 MB/s | 255 kB 00:00 2023-12-12T09:49:44.293 INFO:teuthology.orchestra.run.smithi072.stdout:(140/149): libacl-2.3.1-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2023-12-12T09:49:44.343 INFO:teuthology.orchestra.run.smithi072.stdout:(141/149): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2023-12-12T09:49:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:(148/149): openssl-libs-3.0.7-25.el9.x86_64.rpm 1.4 MB/s | 2.2 MB 00:01 2023-12-12T09:49:44.669 INFO:teuthology.orchestra.run.smithi072.stdout:(142/149): librbd1-19.0.0-96.gf1fbf3c5.el9.x86_ 4.0 MB/s | 3.0 MB 00:00 2023-12-12T09:49:44.986 INFO:teuthology.orchestra.run.smithi072.stdout:(143/149): librados2-19.0.0-96.gf1fbf3c5.el9.x8 3.0 MB/s | 3.3 MB 00:01 2023-12-12T09:49:45.311 INFO:teuthology.orchestra.run.smithi072.stdout:(144/149): NetworkManager-team-1.45.7-1.el9.x86 133 kB/s | 43 kB 00:00 2023-12-12T09:49:45.462 INFO:teuthology.orchestra.run.smithi072.stdout:(145/149): NetworkManager-libnm-1.45.7-1.el9.x8 2.3 MB/s | 1.8 MB 00:00 2023-12-12T09:49:45.834 INFO:teuthology.orchestra.run.smithi072.stdout:(146/149): NetworkManager-1.45.7-1.el9.x86_64.r 1.5 MB/s | 2.3 MB 00:01 2023-12-12T09:49:45.871 INFO:teuthology.orchestra.run.smithi072.stdout:(147/149): NetworkManager-tui-1.45.7-1.el9.x86_ 443 kB/s | 246 kB 00:00 2023-12-12T09:49:45.939 INFO:teuthology.orchestra.run.smithi072.stdout:(148/149): openssl-3.0.7-25.el9.x86_64.rpm 2.6 MB/s | 1.2 MB 00:00 2023-12-12T09:49:46.097 INFO:teuthology.orchestra.run.smithi072.stdout:(149/149): openssl-libs-3.0.7-25.el9.x86_64.rpm 10 MB/s | 2.2 MB 00:00 2023-12-12T09:49:46.111 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:49:46.111 INFO:teuthology.orchestra.run.smithi072.stdout:Total 10 MB/s | 204 MB 00:19 2023-12-12T09:49:46.521 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-12-12T09:49:46.800 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-12-12T09:49:46.800 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-12-12T09:49:49.016 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-12-12T09:49:49.016 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-12-12T09:49:50.664 INFO:teuthology.orchestra.run.smithi113.stdout:(149/149): NetworkManager-libnm-1.45.7-1.el9.x8 193 kB/s | 1.8 MB 00:09 2023-12-12T09:49:50.672 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:49:50.673 INFO:teuthology.orchestra.run.smithi113.stdout:Total 8.1 MB/s | 204 MB 00:25 2023-12-12T09:49:51.076 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2023-12-12T09:49:51.355 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2023-12-12T09:49:51.355 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2023-12-12T09:49:53.593 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2023-12-12T09:49:53.593 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2023-12-12T09:49:55.734 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-12-12T09:49:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/164 2023-12-12T09:49:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/164 2023-12-12T09:49:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/164 2023-12-12T09:49:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/164 2023-12-12T09:49:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2023-12-12T09:49:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2023-12-12T09:49:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2023-12-12T09:49:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 8/164 2023-12-12T09:49:57.186 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 8/164 2023-12-12T09:49:57.215 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 9/164 2023-12-12T09:49:57.365 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 9/164 2023-12-12T09:49:57.399 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:19.0.0-96.gf1fbf3c5.el9.x86_64 10/164 2023-12-12T09:49:57.478 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/164 2023-12-12T09:49:57.814 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/164 2023-12-12T09:49:57.876 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/164 2023-12-12T09:49:57.904 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el9.x86_64 14/164 2023-12-12T09:49:58.053 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el9.x86_64 14/164 2023-12-12T09:49:58.081 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el9.x86_6 15/164 2023-12-12T09:49:58.718 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el9.x86_6 15/164 2023-12-12T09:49:58.807 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/164 2023-12-12T09:49:58.906 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/164 2023-12-12T09:49:59.168 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/164 2023-12-12T09:49:59.214 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/164 2023-12-12T09:49:59.524 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/164 2023-12-12T09:49:59.690 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/164 2023-12-12T09:49:59.738 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/164 2023-12-12T09:49:59.862 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/164 2023-12-12T09:49:59.915 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2023-12-12T09:49:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/164 2023-12-12T09:49:59.962 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/164 2023-12-12T09:50:00.031 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/164 2023-12-12T09:50:00.093 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/164 2023-12-12T09:50:00.126 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/164 2023-12-12T09:50:00.163 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/164 2023-12-12T09:50:00.204 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/164 2023-12-12T09:50:00.250 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/164 2023-12-12T09:50:00.250 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/164 2023-12-12T09:50:00.286 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/164 2023-12-12T09:50:00.310 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/164 2023-12-12T09:50:00.347 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2023-12-12T09:50:00.400 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2023-12-12T09:50:00.441 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x8 29/164 2023-12-12T09:50:00.445 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2023-12-12T09:50:00.500 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el9. 30/164 2023-12-12T09:50:00.617 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el9.x86_64 31/164 2023-12-12T09:50:00.815 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/164 2023-12-12T09:50:00.859 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2023-12-12T09:50:00.878 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/164 2023-12-12T09:50:00.920 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 8/164 2023-12-12T09:50:01.061 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 8/164 2023-12-12T09:50:01.095 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 9/164 2023-12-12T09:50:01.226 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 9/164 2023-12-12T09:50:01.260 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:19.0.0-96.gf1fbf3c5.el9.x86_64 10/164 2023-12-12T09:50:01.320 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/164 2023-12-12T09:50:01.573 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/164 2023-12-12T09:50:01.597 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/164 2023-12-12T09:50:01.659 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/164 2023-12-12T09:50:01.670 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/164 2023-12-12T09:50:01.684 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el9.x86_64 14/164 2023-12-12T09:50:01.725 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/164 2023-12-12T09:50:01.782 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/164 2023-12-12T09:50:01.832 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 38/164 2023-12-12T09:50:01.835 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el9.x86_64 14/164 2023-12-12T09:50:01.868 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el9.x86_6 15/164 2023-12-12T09:50:01.913 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/164 2023-12-12T09:50:02.194 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el9.x86_6 15/164 2023-12-12T09:50:02.287 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/164 2023-12-12T09:50:02.339 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/164 2023-12-12T09:50:02.364 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/164 2023-12-12T09:50:02.452 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/164 2023-12-12T09:50:02.506 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/164 2023-12-12T09:50:02.554 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 43/164 2023-12-12T09:50:02.629 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/164 2023-12-12T09:50:02.632 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/164 2023-12-12T09:50:02.692 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/164 2023-12-12T09:50:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/164 2023-12-12T09:50:02.765 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/164 2023-12-12T09:50:02.812 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/164 2023-12-12T09:50:02.882 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/164 2023-12-12T09:50:02.947 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/164 2023-12-12T09:50:03.017 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/164 2023-12-12T09:50:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/164 2023-12-12T09:50:03.188 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/164 2023-12-12T09:50:03.208 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/164 2023-12-12T09:50:03.223 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/164 2023-12-12T09:50:03.261 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/164 2023-12-12T09:50:03.315 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/164 2023-12-12T09:50:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/164 2023-12-12T09:50:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/164 2023-12-12T09:50:03.434 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/164 2023-12-12T09:50:03.443 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/164 2023-12-12T09:50:03.476 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/164 2023-12-12T09:50:03.501 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/164 2023-12-12T09:50:03.519 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/164 2023-12-12T09:50:03.555 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/164 2023-12-12T09:50:03.556 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/164 2023-12-12T09:50:03.658 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/164 2023-12-12T09:50:03.704 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/164 2023-12-12T09:50:03.708 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 58/164 2023-12-12T09:50:03.738 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/164 2023-12-12T09:50:03.769 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/164 2023-12-12T09:50:03.847 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/164 2023-12-12T09:50:03.851 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2023-12-12T09:50:03.902 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x8 29/164 2023-12-12T09:50:03.932 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/164 2023-12-12T09:50:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el9. 30/164 2023-12-12T09:50:04.068 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el9.x86_64 31/164 2023-12-12T09:50:04.188 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/164 2023-12-12T09:50:04.255 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/164 2023-12-12T09:50:04.279 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/164 2023-12-12T09:50:04.362 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/164 2023-12-12T09:50:04.431 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/164 2023-12-12T09:50:04.484 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/164 2023-12-12T09:50:04.510 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/164 2023-12-12T09:50:04.569 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/164 2023-12-12T09:50:04.617 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/164 2023-12-12T09:50:04.656 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/164 2023-12-12T09:50:04.699 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/164 2023-12-12T09:50:04.803 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/164 2023-12-12T09:50:04.877 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/164 2023-12-12T09:50:04.930 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2023-12-12T09:50:05.069 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2023-12-12T09:50:05.098 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/164 2023-12-12T09:50:05.105 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/164 2023-12-12T09:50:05.149 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/164 2023-12-12T09:50:05.211 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/164 2023-12-12T09:50:05.217 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/164 2023-12-12T09:50:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/164 2023-12-12T09:50:05.270 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/164 2023-12-12T09:50:05.335 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/164 2023-12-12T09:50:05.383 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 38/164 2023-12-12T09:50:05.400 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/164 2023-12-12T09:50:05.438 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/164 2023-12-12T09:50:05.450 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/164 2023-12-12T09:50:05.590 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2023-12-12T09:50:05.646 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/164 2023-12-12T09:50:05.653 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2023-12-12T09:50:05.713 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/164 2023-12-12T09:50:06.250 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/164 2023-12-12T09:50:06.293 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 43/164 2023-12-12T09:50:06.364 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/164 2023-12-12T09:50:06.466 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/164 2023-12-12T09:50:06.511 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/164 2023-12-12T09:50:06.550 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/164 2023-12-12T09:50:06.608 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2023-12-12T09:50:06.617 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/164 2023-12-12T09:50:06.675 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2023-12-12T09:50:06.682 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/164 2023-12-12T09:50:06.708 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2023-12-12T09:50:06.742 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/164 2023-12-12T09:50:06.743 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2023-12-12T09:50:06.931 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/164 2023-12-12T09:50:06.975 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/164 2023-12-12T09:50:07.019 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/164 2023-12-12T09:50:07.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/164 2023-12-12T09:50:07.058 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/164 2023-12-12T09:50:07.144 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/164 2023-12-12T09:50:07.159 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/164 2023-12-12T09:50:07.218 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/164 2023-12-12T09:50:07.271 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/164 2023-12-12T09:50:07.308 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/164 2023-12-12T09:50:07.466 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 58/164 2023-12-12T09:50:07.519 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/164 2023-12-12T09:50:07.588 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/164 2023-12-12T09:50:07.671 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/164 2023-12-12T09:50:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/164 2023-12-12T09:50:07.951 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/164 2023-12-12T09:50:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/164 2023-12-12T09:50:08.103 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/164 2023-12-12T09:50:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/164 2023-12-12T09:50:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/164 2023-12-12T09:50:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/164 2023-12-12T09:50:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/164 2023-12-12T09:50:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/164 2023-12-12T09:50:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/164 2023-12-12T09:50:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/164 2023-12-12T09:50:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/164 2023-12-12T09:50:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/164 2023-12-12T09:50:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2023-12-12T09:50:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2023-12-12T09:50:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/164 2023-12-12T09:50:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/164 2023-12-12T09:50:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/164 2023-12-12T09:50:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/164 2023-12-12T09:50:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/164 2023-12-12T09:50:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/164 2023-12-12T09:50:09.204 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2023-12-12T09:50:09.292 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2023-12-12T09:50:09.631 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/164 2023-12-12T09:50:09.744 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/164 2023-12-12T09:50:09.783 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/164 2023-12-12T09:50:09.828 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : jq-1.6-16.el9.x86_64 95/164 2023-12-12T09:50:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-5.4.4-4.el9.x86_64 96/164 2023-12-12T09:50:10.036 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : luarocks-3.9.2-1.el9.noarch 97/164 2023-12-12T09:50:10.078 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2023-12-12T09:50:10.122 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 99/164 2023-12-12T09:50:10.274 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2023-12-12T09:50:10.341 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2023-12-12T09:50:10.374 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2023-12-12T09:50:10.408 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2023-12-12T09:50:10.509 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2023-12-12T09:50:10.544 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 101/164 2023-12-12T09:50:10.655 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/164 2023-12-12T09:50:10.668 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 101/164 2023-12-12T09:50:10.687 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/164 2023-12-12T09:50:10.703 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 102/164 2023-12-12T09:50:10.758 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2023-12-12T09:50:10.780 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/164 2023-12-12T09:50:11.274 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 104/164 2023-12-12T09:50:11.309 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 105/164 2023-12-12T09:50:11.393 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 105/164 2023-12-12T09:50:11.421 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 106/164 2023-12-12T09:50:11.581 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/164 2023-12-12T09:50:11.722 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/164 2023-12-12T09:50:13.277 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/164 2023-12-12T09:50:13.400 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/164 2023-12-12T09:50:13.439 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/164 2023-12-12T09:50:13.501 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : jq-1.6-16.el9.x86_64 95/164 2023-12-12T09:50:13.622 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-5.4.4-4.el9.x86_64 96/164 2023-12-12T09:50:13.670 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : luarocks-3.9.2-1.el9.noarch 97/164 2023-12-12T09:50:13.703 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2023-12-12T09:50:13.738 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 99/164 2023-12-12T09:50:13.854 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 107/164 2023-12-12T09:50:13.892 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 107/164 2023-12-12T09:50:14.117 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2023-12-12T09:50:14.152 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 101/164 2023-12-12T09:50:14.273 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 101/164 2023-12-12T09:50:14.308 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 102/164 2023-12-12T09:50:14.382 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2023-12-12T09:50:14.524 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 107/164 2023-12-12T09:50:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:19.0.0-96.gf1fbf3c5.el9.x86_64 108/164 2023-12-12T09:50:14.758 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:19.0.0-96.gf1fbf3c5.el9.x86_64 108/164 2023-12-12T09:50:14.758 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-12T09:50:14.758 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-12T09:50:14.758 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:14.802 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el9.x86_64 109/164 2023-12-12T09:50:14.905 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 104/164 2023-12-12T09:50:14.938 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 105/164 2023-12-12T09:50:15.030 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 105/164 2023-12-12T09:50:15.057 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 106/164 2023-12-12T09:50:17.664 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 107/164 2023-12-12T09:50:17.703 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 107/164 2023-12-12T09:50:18.336 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 107/164 2023-12-12T09:50:18.383 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:19.0.0-96.gf1fbf3c5.el9.x86_64 108/164 2023-12-12T09:50:18.583 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:19.0.0-96.gf1fbf3c5.el9.x86_64 108/164 2023-12-12T09:50:18.583 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-12T09:50:18.583 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-12T09:50:18.583 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:18.643 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el9.x86_64 109/164 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el9.x86_64 109/164 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /root 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2023-12-12T09:50:30.735 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:30.950 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:19.0.0-96.gf1fbf3c5.el9.noarch 110/164 2023-12-12T09:50:31.001 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:19.0.0-96.gf1fbf3c5.el9.noarch 110/164 2023-12-12T09:50:31.002 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:31.002 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-12T09:50:31.002 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:31.323 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el9.x86_64 109/164 2023-12-12T09:50:31.324 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2023-12-12T09:50:31.324 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2023-12-12T09:50:31.324 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2023-12-12T09:50:31.324 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2023-12-12T09:50:31.324 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2023-12-12T09:50:31.324 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2023-12-12T09:50:31.324 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2023-12-12T09:50:31.324 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:19.0.0-96.gf1fbf3c5.el9.noarch 110/164 2023-12-12T09:50:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:19.0.0-96.gf1fbf3c5.el9.noarch 110/164 2023-12-12T09:50:31.595 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:31.595 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-12T09:50:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:32.678 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 111/164 2023-12-12T09:50:32.765 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 111/164 2023-12-12T09:50:32.765 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:32.765 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-12T09:50:32.765 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-12T09:50:32.765 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-12T09:50:32.765 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:33.038 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:19.0.0-96.gf1fbf3c5.el9.x86_64 112/164 2023-12-12T09:50:33.139 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:19.0.0-96.gf1fbf3c5.el9.x86_64 112/164 2023-12-12T09:50:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-12T09:50:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-12T09:50:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-12T09:50:33.139 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:33.325 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 111/164 2023-12-12T09:50:33.414 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 111/164 2023-12-12T09:50:33.414 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:33.414 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-12T09:50:33.414 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-12T09:50:33.414 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-12T09:50:33.414 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:33.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:19.0.0-96.gf1fbf3c5.el9.x86_64 113/164 2023-12-12T09:50:33.642 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:19.0.0-96.gf1fbf3c5.el9.x86_64 112/164 2023-12-12T09:50:33.707 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:19.0.0-96.gf1fbf3c5.el9.x86_64 113/164 2023-12-12T09:50:33.707 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:33.707 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-12T09:50:33.707 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-12T09:50:33.707 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-12T09:50:33.707 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:33.711 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:19.0.0-96.gf1fbf3c5.el9.x86_64 112/164 2023-12-12T09:50:33.711 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:33.711 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-12T09:50:33.711 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-12T09:50:33.711 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-12T09:50:33.712 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:33.818 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2023-12-12T09:50:34.209 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:19.0.0-96.gf1fbf3c5.el9.x86_64 113/164 2023-12-12T09:50:34.291 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:19.0.0-96.gf1fbf3c5.el9.x86_64 113/164 2023-12-12T09:50:34.291 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:34.291 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-12T09:50:34.291 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-12T09:50:34.291 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-12T09:50:34.291 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:34.411 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2023-12-12T09:50:34.453 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 115/164 2023-12-12T09:50:34.482 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 115/164 2023-12-12T09:50:34.517 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el9 116/164 2023-12-12T09:50:34.608 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 117/164 2023-12-12T09:50:34.638 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarc 118/164 2023-12-12T09:50:34.947 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarc 118/164 2023-12-12T09:50:34.990 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el9.noa 119/164 2023-12-12T09:50:35.021 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 115/164 2023-12-12T09:50:35.062 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 115/164 2023-12-12T09:50:35.098 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el9 116/164 2023-12-12T09:50:35.176 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 117/164 2023-12-12T09:50:35.208 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarc 118/164 2023-12-12T09:50:35.526 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarc 118/164 2023-12-12T09:50:35.567 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el9.noa 119/164 2023-12-12T09:50:36.114 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el9.noa 119/164 2023-12-12T09:50:36.149 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 120/164 2023-12-12T09:50:36.294 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 120/164 2023-12-12T09:50:36.468 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el9. 121/164 2023-12-12T09:50:36.496 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el9.x86_64 122/164 2023-12-12T09:50:36.566 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el9.x86_64 122/164 2023-12-12T09:50:36.566 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:36.567 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-12T09:50:36.567 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-12T09:50:36.567 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-12T09:50:36.567 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:36.622 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el9.noarch 123/164 2023-12-12T09:50:36.662 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el9.noarch 123/164 2023-12-12T09:50:36.695 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el9.noa 119/164 2023-12-12T09:50:36.725 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 120/164 2023-12-12T09:50:36.753 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:19.0.0-96.gf1fbf3c5.el9.x86_64 124/164 2023-12-12T09:50:36.868 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 120/164 2023-12-12T09:50:37.034 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el9. 121/164 2023-12-12T09:50:37.069 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el9.x86_64 122/164 2023-12-12T09:50:37.151 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el9.x86_64 122/164 2023-12-12T09:50:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-12T09:50:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-12T09:50:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-12T09:50:37.152 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:37.220 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el9.noarch 123/164 2023-12-12T09:50:37.259 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el9.noarch 123/164 2023-12-12T09:50:37.331 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:19.0.0-96.gf1fbf3c5.el9.x86_64 124/164 2023-12-12T09:50:37.649 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libtool-2.4.6-45.el9.x86_64 125/164 2023-12-12T09:50:37.686 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 126/164 2023-12-12T09:50:37.758 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 126/164 2023-12-12T09:50:37.758 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:37.758 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-12T09:50:37.758 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-12T09:50:37.758 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-12T09:50:37.758 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:37.812 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 127/164 2023-12-12T09:50:37.884 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 127/164 2023-12-12T09:50:37.884 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:37.884 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-12T09:50:37.884 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:37.947 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 128/164 2023-12-12T09:50:38.023 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 128/164 2023-12-12T09:50:38.024 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:38.024 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-12T09:50:38.024 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-12T09:50:38.024 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-12T09:50:38.024 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:38.172 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 125/164 2023-12-12T09:50:38.206 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 126/164 2023-12-12T09:50:38.287 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 126/164 2023-12-12T09:50:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-12T09:50:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-12T09:50:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-12T09:50:38.288 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:38.331 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 127/164 2023-12-12T09:50:38.420 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 127/164 2023-12-12T09:50:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-12T09:50:38.421 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:38.482 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 128/164 2023-12-12T09:50:38.563 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 128/164 2023-12-12T09:50:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-12T09:50:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-12T09:50:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-12T09:50:38.563 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:38.607 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 129/164 2023-12-12T09:50:38.688 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 129/164 2023-12-12T09:50:38.688 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:38.688 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-12T09:50:38.688 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-12T09:50:38.688 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-12T09:50:38.688 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:38.949 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 129/164 2023-12-12T09:50:39.019 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 129/164 2023-12-12T09:50:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-12T09:50:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-12T09:50:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-12T09:50:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-12T09:50:39.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:42.902 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:19.0.0-96.gf1fbf3c5.el9.x86_64 130/164 2023-12-12T09:50:42.958 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 131/164 2023-12-12T09:50:43.023 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 132/164 2023-12-12T09:50:43.060 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 133/164 2023-12-12T09:50:43.172 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:19.0.0-96.gf1fbf3c5.el9.x86_64 130/164 2023-12-12T09:50:43.228 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 131/164 2023-12-12T09:50:43.281 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 132/164 2023-12-12T09:50:43.320 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 133/164 2023-12-12T09:50:43.340 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 134/164 2023-12-12T09:50:43.487 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 135/164 2023-12-12T09:50:43.521 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 136/164 2023-12-12T09:50:43.564 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 137/164 2023-12-12T09:50:43.613 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 134/164 2023-12-12T09:50:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 138/164 2023-12-12T09:50:43.701 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/164 2023-12-12T09:50:43.738 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/164 2023-12-12T09:50:43.742 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 135/164 2023-12-12T09:50:43.774 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 136/164 2023-12-12T09:50:43.789 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 141/164 2023-12-12T09:50:43.822 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 137/164 2023-12-12T09:50:43.839 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 142/164 2023-12-12T09:50:43.931 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 138/164 2023-12-12T09:50:44.005 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/164 2023-12-12T09:50:44.044 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/164 2023-12-12T09:50:44.098 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 141/164 2023-12-12T09:50:44.144 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 142/164 2023-12-12T09:50:44.752 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 143/164 2023-12-12T09:50:44.814 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 143/164 2023-12-12T09:50:44.874 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 143/164 2023-12-12T09:50:45.064 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 143/164 2023-12-12T09:50:45.116 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 143/164 2023-12-12T09:50:45.182 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 143/164 2023-12-12T09:50:46.210 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-top-2:19.0.0-96.gf1fbf3c5.el9.noarch 144/164 2023-12-12T09:50:46.401 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 145/164 2023-12-12T09:50:46.465 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 146/164 2023-12-12T09:50:46.504 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:19.0.0-96.gf1fbf3c5.el9.noarch 144/164 2023-12-12T09:50:46.553 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 147/164 2023-12-12T09:50:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 145/164 2023-12-12T09:50:46.687 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 146/164 2023-12-12T09:50:46.709 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flex-2.6.4-9.el9.x86_64 148/164 2023-12-12T09:50:46.758 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : bison-3.7.4-5.el9.x86_64 149/164 2023-12-12T09:50:46.759 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 147/164 2023-12-12T09:50:46.782 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 150/164 2023-12-12T09:50:46.819 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 151/164 2023-12-12T09:50:46.861 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 152/164 2023-12-12T09:50:46.861 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/164 2023-12-12T09:50:46.911 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 148/164 2023-12-12T09:50:46.942 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 149/164 2023-12-12T09:50:46.975 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 150/164 2023-12-12T09:50:47.001 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 151/164 2023-12-12T09:50:47.003 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/164 2023-12-12T09:50:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 152/164 2023-12-12T09:50:47.043 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/164 2023-12-12T09:50:47.054 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 154/164 2023-12-12T09:50:47.109 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 155/164 2023-12-12T09:50:47.150 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 156/164 2023-12-12T09:50:47.150 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 157/164 2023-12-12T09:50:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/164 2023-12-12T09:50:47.226 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 154/164 2023-12-12T09:50:47.273 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 155/164 2023-12-12T09:50:47.288 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 157/164 2023-12-12T09:50:47.305 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 156/164 2023-12-12T09:50:47.305 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 157/164 2023-12-12T09:50:47.327 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 158/164 2023-12-12T09:50:47.367 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 159/164 2023-12-12T09:50:47.367 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 159/164 2023-12-12T09:50:47.463 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 157/164 2023-12-12T09:50:47.498 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 158/164 2023-12-12T09:50:47.537 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 159/164 2023-12-12T09:50:47.538 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 159/164 2023-12-12T09:50:47.704 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 159/164 2023-12-12T09:50:47.704 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/164 2023-12-12T09:50:47.847 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/164 2023-12-12T09:50:47.872 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 159/164 2023-12-12T09:50:47.872 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/164 2023-12-12T09:50:47.892 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 161/164 2023-12-12T09:50:47.927 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 162/164 2023-12-12T09:50:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 163/164 2023-12-12T09:50:48.015 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/164 2023-12-12T09:50:48.051 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 161/164 2023-12-12T09:50:48.084 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 162/164 2023-12-12T09:50:48.090 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 164/164 2023-12-12T09:50:48.118 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 163/164 2023-12-12T09:50:48.236 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 164/164 2023-12-12T09:50:48.351 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 164/164 2023-12-12T09:50:48.351 INFO:teuthology.orchestra.run.smithi072.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-12T09:50:48.351 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:48.506 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 164/164 2023-12-12T09:50:48.506 INFO:teuthology.orchestra.run.smithi113.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-12T09:50:48.506 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:53.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:19.0.0-96.gf1fbf3c5.el9.x86_64 1/164 2023-12-12T09:50:53.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2/164 2023-12-12T09:50:53.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 3/164 2023-12-12T09:50:53.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 4/164 2023-12-12T09:50:53.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 5/164 2023-12-12T09:50:53.319 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:19.0.0-96.gf1fbf3c5.el9.x86_64 6/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el9.x86_64 7/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:19.0.0-96.gf1fbf3c5.el9.x86_64 8/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 9/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 10/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el9.x86_64 11/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:19.0.0-96.gf1fbf3c5.el9.x86_64 12/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 13/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 14/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 15/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el9.x86_64 16/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 17/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el9.x86_6 18/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 19/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el9. 20/164 2023-12-12T09:50:53.320 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x8 21/164 2023-12-12T09:50:53.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el9.x86_64 22/164 2023-12-12T09:50:53.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:19.0.0-96.gf1fbf3c5.el9.x86_64 23/164 2023-12-12T09:50:53.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 24/164 2023-12-12T09:50:53.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 25/164 2023-12-12T09:50:53.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 26/164 2023-12-12T09:50:53.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 27/164 2023-12-12T09:50:53.321 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 28/164 2023-12-12T09:50:53.332 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 29/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarc 30/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el9.noa 31/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 32/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el9. 33/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el9.noarch 34/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el9 35/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:19.0.0-96.gf1fbf3c5.el9.noarch 36/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 37/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:19.0.0-96.gf1fbf3c5.el9.noarch 38/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/164 2023-12-12T09:50:53.333 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : jq-1.6-16.el9.x86_64 47/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/164 2023-12-12T09:50:53.334 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 57/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 58/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 59/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/164 2023-12-12T09:50:53.335 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/164 2023-12-12T09:50:53.336 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 84/164 2023-12-12T09:50:53.337 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 85/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 86/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 87/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 92/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 93/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 94/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/164 2023-12-12T09:50:53.338 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 97/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 98/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 101/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 103/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/164 2023-12-12T09:50:53.339 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 109/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 115/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 116/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 117/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 118/164 2023-12-12T09:50:53.340 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 119/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 120/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 121/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 122/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 124/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 125/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 126/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 127/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 128/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 129/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 130/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 131/164 2023-12-12T09:50:53.341 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 132/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 133/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 134/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 135/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 136/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 137/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 138/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 139/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 140/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-4.el9.x86_64 141/164 2023-12-12T09:50:53.342 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-3.el9.x86_64 142/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 143/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 144/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 145/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 146/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 147/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 148/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 153/164 2023-12-12T09:50:53.343 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 154/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 155/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 156/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 157/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 158/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 159/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 160/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 161/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 162/164 2023-12-12T09:50:53.344 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 163/164 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 164/164 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout: acl-2.3.1-4.el9.x86_64 2023-12-12T09:50:55.416 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.417 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.418 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-12T09:50:55.419 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: jq-1.6-16.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-10.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-10.el9.noarch 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-12T09:50:55.420 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: lua-5.4.4-4.el9.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks-3.9.2-1.el9.noarch 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-12T09:50:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-10.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.11-1.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-12T09:50:55.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-12T09:50:55.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-12T09:50:55.424 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.425 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2023-12-12T09:50:55.426 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-12T09:50:55.427 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-12T09:50:55.427 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-12T09:50:55.427 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-12T09:50:55.427 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-12T09:50:55.427 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-12T09:50:55.427 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:50:55.427 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2023-12-12T09:50:56.037 DEBUG:teuthology.parallel:result is None 2023-12-12T09:50:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:19.0.0-96.gf1fbf3c5.el9.x86_64 1/164 2023-12-12T09:50:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 3/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 4/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 5/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:19.0.0-96.gf1fbf3c5.el9.x86_64 6/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el9.x86_64 7/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:19.0.0-96.gf1fbf3c5.el9.x86_64 8/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 9/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 10/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el9.x86_64 11/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:19.0.0-96.gf1fbf3c5.el9.x86_64 12/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 13/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 14/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 15/164 2023-12-12T09:50:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el9.x86_64 16/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 17/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el9.x86_6 18/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 19/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el9. 20/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x8 21/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el9.x86_64 22/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:19.0.0-96.gf1fbf3c5.el9.x86_64 23/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 24/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 25/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 26/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 27/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 28/164 2023-12-12T09:50:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 29/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarc 30/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el9.noa 31/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 32/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el9. 33/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el9.noarch 34/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el9 35/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:19.0.0-96.gf1fbf3c5.el9.noarch 36/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 37/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-top-2:19.0.0-96.gf1fbf3c5.el9.noarch 38/164 2023-12-12T09:50:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : jq-1.6-16.el9.x86_64 47/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/164 2023-12-12T09:50:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 57/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 58/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 59/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/164 2023-12-12T09:50:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/164 2023-12-12T09:50:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 84/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 85/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 86/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 87/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 92/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 93/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 94/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/164 2023-12-12T09:50:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 97/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 98/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 101/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 103/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 109/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/164 2023-12-12T09:50:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 115/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 116/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 117/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 118/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 119/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 120/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 121/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 122/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 124/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 125/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 126/164 2023-12-12T09:50:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 127/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 128/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 129/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 130/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 131/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 132/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : re2-1:20211101-3.el9.x86_64 133/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 134/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 135/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 136/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 137/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 138/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 139/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 140/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : acl-2.3.1-4.el9.x86_64 141/164 2023-12-12T09:50:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : acl-2.3.1-3.el9.x86_64 142/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 143/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 144/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 145/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 146/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 147/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 148/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 153/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 154/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 155/164 2023-12-12T09:50:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 156/164 2023-12-12T09:50:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 157/164 2023-12-12T09:50:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 158/164 2023-12-12T09:50:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 159/164 2023-12-12T09:50:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 160/164 2023-12-12T09:50:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 161/164 2023-12-12T09:50:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 162/164 2023-12-12T09:50:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 163/164 2023-12-12T09:50:58.973 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 164/164 2023-12-12T09:50:58.973 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:58.973 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2023-12-12T09:50:58.973 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-12T09:50:58.973 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-12T09:50:58.973 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-12T09:50:58.973 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-12T09:50:58.973 INFO:teuthology.orchestra.run.smithi072.stdout: acl-2.3.1-4.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: automake-1.16.2-8.el9.noarch 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: bison-3.7.4-5.el9.x86_64 2023-12-12T09:50:58.974 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.975 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top-2:19.0.0-96.gf1fbf3c5.el9.noarch 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: flex-2.6.4-9.el9.x86_64 2023-12-12T09:50:58.976 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: jq-1.6-16.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-9.0.0-10.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc-9.0.0-10.el9.noarch 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-12T09:50:58.977 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: lua-5.4.4-4.el9.x86_64 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks-3.9.2-1.el9.noarch 2023-12-12T09:50:58.978 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs-9.0.0-10.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-13.11-1.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.979 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-12T09:50:58.980 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-12T09:50:58.981 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply-3.11-14.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.982 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:19.0.0-96.gf1fbf3c5.el9.x86_64 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: re2-1:20211101-3.el9.x86_64 2023-12-12T09:50:58.983 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-12T09:50:58.984 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-12T09:50:58.984 INFO:teuthology.orchestra.run.smithi072.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-12T09:50:58.984 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-12T09:50:58.984 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-12T09:50:58.984 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-12T09:50:58.984 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:50:58.984 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-12-12T09:50:59.732 DEBUG:teuthology.parallel:result is None 2023-12-12T09:50:59.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:50:59.950 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-12T09:51:00.011 INFO:teuthology.orchestra.run.smithi072.stdout:19.0.0-96.gf1fbf3c5.el9 2023-12-12T09:51:00.012 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-96.gf1fbf3c5.el9 2023-12-12T09:51:00.012 INFO:teuthology.task.install:The correct ceph version 19.0.0-96.gf1fbf3c5 is installed. 2023-12-12T09:51:00.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:51:00.152 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-12T09:51:00.208 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-96.gf1fbf3c5.el9 2023-12-12T09:51:00.208 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-96.gf1fbf3c5.el9 2023-12-12T09:51:00.209 INFO:teuthology.task.install:The correct ceph version 19.0.0-96.gf1fbf3c5 is installed. 2023-12-12T09:51:00.210 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-12T09:51:00.210 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:00.211 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-12T09:51:00.292 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:00.292 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-12T09:51:00.370 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-12T09:51:00.371 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:00.371 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2023-12-12T09:51:00.445 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-12T09:51:00.559 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:00.559 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2023-12-12T09:51:00.637 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-12T09:51:00.751 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-12T09:51:00.751 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:00.751 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-12T09:51:00.827 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-12T09:51:00.940 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:00.940 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-12T09:51:01.010 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-12T09:51:01.138 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-12T09:51:01.139 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:01.139 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2023-12-12T09:51:01.208 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-12T09:51:01.319 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:01.319 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2023-12-12T09:51:01.391 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-12T09:51:01.504 INFO:teuthology.run_tasks:Running task ceph... 2023-12-12T09:51:01.576 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-12T09:51:01.576 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /var/log/ceph 2023-12-12T09:51:01.581 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2023-12-12T09:51:01.657 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-12T09:51:01.658 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-12T09:51:01.662 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-12T09:51:01.740 INFO:tasks.ceph:Creating extra log directories... 2023-12-12T09:51:01.740 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-12T09:51:01.745 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-12T09:51:01.836 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-12T09:51:01.836 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-12T09:51:01.836 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-12T09:51:01.836 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:01.836 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-12T09:51:01.873 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-12T09:51:02.080 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-12T09:51:02.150 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:02.150 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-12T09:51:02.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-12T09:51:02.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-12T09:51:02.539 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-12T09:51:02.539 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-12T09:51:02.540 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219', 'branch': 'wip-mchangir-testing-20231205.064351', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7488219', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing-20231205.064351'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing-20231205.064351', 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'}}, 'owner': 'mchangir', 'priority': 1000, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5473, 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'sleep_before_teardown': 0, 'subset': '600/1024', 'suite': 'fs', 'suite_branch': 'wip-mchangir-testing-20231205.064351', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi072.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEU3ThwljGX0CbUGQkQeI+yI9VjYeXL1STRtECiXaiXc61gFHpsM3Tixc+RkxyInCiWcLjawjrMcsJUQgPtJVZU=', 'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6xr8M/tal4s4/lDEJJ6DdIoRSlvP3w48DiCcXXkVGWpUf2hlO+7UPcu9si7kAnZ4/agxdjzwt+2abI7ruRLJg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_strays']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73', 'timestamp': '2023-12-12_08:46:52', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-12-12T09:51:02.540 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-12T09:51:02.546 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-12T09:51:02.604 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/run/ceph 2023-12-12T09:51:02.634 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2023-12-12T09:51:02.731 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:02.731 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2023-12-12T09:51:02.771 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-12T09:51:02.771 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2023-12-12T09:51:02.853 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-12T09:51:02.854 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:02.854 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-12-12T09:51:02.854 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:02.854 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:02.854 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-12-12 09:50:48.835233519 +0000 2023-12-12T09:51:02.854 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-12-12 09:47:53.146756348 +0000 2023-12-12T09:51:02.854 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-12-12 09:47:53.146756348 +0000 2023-12-12T09:51:02.854 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2023-12-12 09:47:53.146756348 +0000 2023-12-12T09:51:02.854 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-12T09:51:02.963 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-12-12T09:51:02.963 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-12-12T09:51:02.963 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000349032 s, 1.5 MB/s 2023-12-12T09:51:02.966 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-12T09:51:03.006 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2023-12-12T09:51:03.104 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-12T09:51:03.125 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:03.125 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 649 Links: 1 2023-12-12T09:51:03.125 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:03.126 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:03.126 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-12-12 09:50:48.836233499 +0000 2023-12-12T09:51:03.126 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-12-12 09:47:54.141735707 +0000 2023-12-12T09:51:03.126 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-12-12 09:47:54.141735707 +0000 2023-12-12T09:51:03.126 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2023-12-12 09:47:54.141735707 +0000 2023-12-12T09:51:03.126 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-12T09:51:03.215 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-12-12T09:51:03.216 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-12-12T09:51:03.216 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000289645 s, 1.8 MB/s 2023-12-12T09:51:03.219 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-12T09:51:03.272 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2023-12-12T09:51:03.317 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-12T09:51:03.317 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:03.317 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 668 Links: 1 2023-12-12T09:51:03.317 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:03.318 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:03.318 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-12-12 09:50:48.836233499 +0000 2023-12-12T09:51:03.318 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-12-12 09:47:55.262712452 +0000 2023-12-12T09:51:03.318 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-12-12 09:47:55.262712452 +0000 2023-12-12T09:51:03.318 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2023-12-12 09:47:55.262712452 +0000 2023-12-12T09:51:03.318 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-12T09:51:03.455 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-12-12T09:51:03.455 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-12-12T09:51:03.455 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000462011 s, 1.1 MB/s 2023-12-12T09:51:03.459 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-12T09:51:03.516 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2023-12-12T09:51:03.565 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-12T09:51:03.566 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:03.566 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-12-12T09:51:03.566 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:03.566 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:03.566 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-12-12 09:50:48.837233479 +0000 2023-12-12T09:51:03.566 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-12-12 09:47:56.366689550 +0000 2023-12-12T09:51:03.566 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-12-12 09:47:56.366689550 +0000 2023-12-12T09:51:03.566 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2023-12-12 09:47:56.366689550 +0000 2023-12-12T09:51:03.566 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-12T09:51:03.656 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-12-12T09:51:03.656 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-12-12T09:51:03.657 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000374218 s, 1.4 MB/s 2023-12-12T09:51:03.660 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-12T09:51:03.719 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:03.719 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:03.719 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2023-12-12T09:51:03.778 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-12T09:51:03.779 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2023-12-12T09:51:03.832 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-12T09:51:03.832 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:03.832 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 631 Links: 1 2023-12-12T09:51:03.832 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:03.833 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:03.833 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2023-12-12 09:50:48.962042788 +0000 2023-12-12T09:51:03.833 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2023-12-12 09:47:45.751276221 +0000 2023-12-12T09:51:03.833 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2023-12-12 09:47:45.751276221 +0000 2023-12-12T09:51:03.833 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2023-12-12 09:47:45.751276221 +0000 2023-12-12T09:51:03.833 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-12T09:51:03.923 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2023-12-12T09:51:03.924 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2023-12-12T09:51:03.924 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000498909 s, 1.0 MB/s 2023-12-12T09:51:03.927 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-12T09:51:03.985 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2023-12-12T09:51:04.038 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-12T09:51:04.038 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:04.038 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-12-12T09:51:04.038 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:04.038 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:04.038 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2023-12-12 09:50:48.963042767 +0000 2023-12-12T09:51:04.039 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2023-12-12 09:47:46.784254494 +0000 2023-12-12T09:51:04.039 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2023-12-12 09:47:46.784254494 +0000 2023-12-12T09:51:04.039 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2023-12-12 09:47:46.784254494 +0000 2023-12-12T09:51:04.039 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-12T09:51:04.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2023-12-12T09:51:04.130 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2023-12-12T09:51:04.130 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000324711 s, 1.6 MB/s 2023-12-12T09:51:04.134 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-12T09:51:04.195 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2023-12-12 09:50:48.963042767 +0000 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2023-12-12 09:47:47.819232724 +0000 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2023-12-12 09:47:47.819232724 +0000 2023-12-12T09:51:04.247 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2023-12-12 09:47:47.819232724 +0000 2023-12-12T09:51:04.247 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-12T09:51:04.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2023-12-12T09:51:04.335 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2023-12-12T09:51:04.335 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000317473 s, 1.6 MB/s 2023-12-12T09:51:04.339 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-12T09:51:04.396 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2023-12-12T09:51:04.446 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-12T09:51:04.447 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:04.447 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-12-12T09:51:04.447 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:04.447 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:04.447 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2023-12-12 09:50:48.964042746 +0000 2023-12-12T09:51:04.447 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2023-12-12 09:47:48.833211397 +0000 2023-12-12T09:51:04.447 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2023-12-12 09:47:48.833211397 +0000 2023-12-12T09:51:04.447 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2023-12-12 09:47:48.833211397 +0000 2023-12-12T09:51:04.448 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-12T09:51:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2023-12-12T09:51:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2023-12-12T09:51:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000310202 s, 1.7 MB/s 2023-12-12T09:51:04.538 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-12T09:51:04.596 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:04.596 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-12T09:51:04.596 INFO:tasks.ceph:Generating config... 2023-12-12T09:51:04.597 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-12T09:51:04.597 INFO:tasks.ceph:[client] debug client = 20 2023-12-12T09:51:04.597 INFO:tasks.ceph:[client] debug ms = 1 2023-12-12T09:51:04.598 INFO:tasks.ceph:[client] pid file = 2023-12-12T09:51:04.598 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-12T09:51:04.598 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-12T09:51:04.598 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-12T09:51:04.598 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-12T09:51:04.599 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-12T09:51:04.599 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-12T09:51:04.599 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-12T09:51:04.599 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-12T09:51:04.599 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-12T09:51:04.599 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-12T09:51:04.600 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-12T09:51:04.600 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-12T09:51:04.600 INFO:tasks.ceph:Setting up mon.a... 2023-12-12T09:51:04.600 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-12T09:51:04.745 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.keyring 2023-12-12T09:51:04.753 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-12T09:51:04.888 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-12T09:51:04.979 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.72'), ('mon.b', '172.21.15.113'), ('mon.c', '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]')] 2023-12-12T09:51:04.979 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.72,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-12T09:51:04.980 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:04.980 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-12T09:51:05.035 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.72 --add b 172.21.15.113 --addv c '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:05.192 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:05.192 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: generated fsid bd07f350-42f2-4d45-9a3d-13364a8566fe 2023-12-12T09:51:05.192 INFO:teuthology.orchestra.run.smithi072.stdout:setting min_mon_release = pacific 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 0 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:fsid bd07f350-42f2-4d45-9a3d-13364a8566fe 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:last_changed 2023-12-12T09:51:05.191469+0000 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:created 2023-12-12T09:51:05.191469+0000 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:min_mon_release 16 (pacific) 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:election_strategy: 1 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:2: [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] mon.c 2023-12-12T09:51:05.193 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-12T09:51:05.195 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-12T09:51:05.251 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bd07f350-42f2-4d45-9a3d-13364a8566fe... 2023-12-12T09:51:05.252 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-12T09:51:05.260 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-12T09:51:05.423 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2023-12-12T09:51:05.423 INFO:teuthology.orchestra.run.smithi072.stdout: chdir = "" 2023-12-12T09:51:05.423 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-12T09:51:05.423 INFO:teuthology.orchestra.run.smithi072.stdout: auth supported = cephx 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: filestore xattr use omap = true 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: mon clock drift allowed = 1.000 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: osd crush chooseleaf type = 0 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: auth debug = true 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on old message = true 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on bug = true 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: mon pg warn max object skew = 0 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.424 INFO:teuthology.orchestra.run.smithi072.stdout: # disable pg_autoscaler by default for new pools 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default size = 2 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow primary affinity = true 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow pg remap = true 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on legacy crush tunables = false 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on crush straw calc version zero = false 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on no sortbitwise = false 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on osd down out interval zero = false 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on too few osds = false 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-12T09:51:05.425 INFO:teuthology.orchestra.run.smithi072.stdout: mon_allow_pool_size_one = true 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: osd default data pool replay window = 5 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: mon allow pool delete = true 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: mon cluster log file level = debug 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: debug asserts on shutdown = true 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: mon health detail to clog = false 2023-12-12T09:51:05.426 INFO:teuthology.orchestra.run.smithi072.stdout: mon host = "172.21.15.72,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on skipped message = False 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = bd07f350-42f2-4d45-9a3d-13364a8566fe 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout:[osd] 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: osd journal size = 100 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub load threshold = 5.0 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub max interval = 600 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: osd mclock profile = high_recovery_ops 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: osd recover clone overlap = true 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: osd recovery max chunk = 1048576 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.427 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug shutdown = true 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug op order = true 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify stray on activate = true 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug trim objects = true 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: osd open classes on start = true 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug pg log writeout = true 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: osd deep scrub update digest min age = 30 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: osd map max advance = 10 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.428 INFO:teuthology.orchestra.run.smithi072.stdout: journal zero on create = true 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: filestore ondisk finisher threads = 3 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: filestore apply finisher threads = 3 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: bdev debug aio = true 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug misdirected ops = true 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore block size = 96636764160 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore fsck on mount = True 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluefs = 20 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluestore = 20 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-12-12T09:51:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: debug osd = 20 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: debug rocksdb = 10 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd backfillfull_ratio = 0.85 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd full ratio = 0.9 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd nearfull ratio = 0.8 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: osd failsafe full ratio = 0.95 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: osd objectstore = bluestore 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: osd op complaint time = 180 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr] 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: debug mgr = 20 2023-12-12T09:51:05.430 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: mgr/telemetry/nag = false 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout:[mon] 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: debug paxos = 20 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: mon data avail warn = 5 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: mon mgr mkfs grace = 240 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd reporter subtree level = osd 2023-12-12T09:51:05.431 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd prime pg temp = true 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: auth mon ticket ttl = 660 # 11m 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: auth service ticket ttl = 240 # 4m 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: # don't complain about insecure global_id in the test suite 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: # 1m isn't quite enough 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: mon_down_mkfs_grace = 2m 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_filestore_osds = false 2023-12-12T09:51:05.432 INFO:teuthology.orchestra.run.smithi072.stdout: mon op complaint time = 120 2023-12-12T09:51:05.433 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:05.433 INFO:teuthology.orchestra.run.smithi072.stdout:[client] 2023-12-12T09:51:05.433 INFO:teuthology.orchestra.run.smithi072.stdout: rgw cache enabled = true 2023-12-12T09:51:05.433 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable ops log = true 2023-12-12T09:51:05.433 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable usage log = true 2023-12-12T09:51:05.433 INFO:teuthology.orchestra.run.smithi072.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-12T09:51:05.433 INFO:teuthology.orchestra.run.smithi072.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-12T09:51:05.433 INFO:teuthology.orchestra.run.smithi072.stdout: client mount timeout = 600 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout: debug client = 20 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = "" 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout: rados mon op timeout = 900 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout: rados osd op timeout = 900 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.a] 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.b] 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.c] 2023-12-12T09:51:05.434 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a] 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c] 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b] 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d] 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout:[mds] 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout: debug mds = 20 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout: debug mds balancer = 20 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout: mds debug frag = True 2023-12-12T09:51:05.435 INFO:teuthology.orchestra.run.smithi072.stdout: mds debug scatterstat = True 2023-12-12T09:51:05.436 INFO:teuthology.orchestra.run.smithi072.stdout: mds op complaint time = 180 2023-12-12T09:51:05.436 INFO:teuthology.orchestra.run.smithi072.stdout: mds verify scatter = True 2023-12-12T09:51:05.436 INFO:teuthology.orchestra.run.smithi072.stdout: osd op complaint time = 180 2023-12-12T09:51:05.436 INFO:teuthology.orchestra.run.smithi072.stdout: rados mon op timeout = 900 2023-12-12T09:51:05.436 INFO:teuthology.orchestra.run.smithi072.stdout: rados osd op timeout = 900 2023-12-12T09:51:05.437 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2023-12-12T09:51:05.437 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2023-12-12T09:51:05.437 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-12T09:51:05.437 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2023-12-12T09:51:05.437 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.438 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2023-12-12T09:51:05.438 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.438 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2023-12-12T09:51:05.438 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.438 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2023-12-12T09:51:05.438 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2023-12-12T09:51:05.439 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.439 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2023-12-12T09:51:05.439 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2023-12-12T09:51:05.439 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.440 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-12T09:51:05.440 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2023-12-12T09:51:05.440 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.440 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2023-12-12T09:51:05.440 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-12T09:51:05.440 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.440 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2023-12-12T09:51:05.440 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.441 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2023-12-12T09:51:05.441 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2023-12-12T09:51:05.441 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2023-12-12T09:51:05.441 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2023-12-12T09:51:05.441 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2023-12-12T09:51:05.441 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2023-12-12T09:51:05.441 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2023-12-12T09:51:05.441 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-12T09:51:05.442 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-12T09:51:05.442 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2023-12-12T09:51:05.442 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.442 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-12T09:51:05.442 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.442 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2023-12-12T09:51:05.442 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.443 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2023-12-12T09:51:05.443 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.443 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2023-12-12T09:51:05.443 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2023-12-12T09:51:05.443 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2023-12-12T09:51:05.443 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.72,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2023-12-12T09:51:05.444 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2023-12-12T09:51:05.444 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = bd07f350-42f2-4d45-9a3d-13364a8566fe 2023-12-12T09:51:05.444 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.444 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2023-12-12T09:51:05.444 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2023-12-12T09:51:05.444 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.444 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2023-12-12T09:51:05.445 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2023-12-12T09:51:05.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 20 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 20 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 10 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2023-12-12T09:51:05.447 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2023-12-12T09:51:05.448 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2023-12-12T09:51:05.449 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2023-12-12T09:51:05.449 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2023-12-12T09:51:05.449 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2023-12-12T09:51:05.449 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.449 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2023-12-12T09:51:05.450 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-12-12T09:51:05.450 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2023-12-12T09:51:05.451 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2023-12-12T09:51:05.451 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2023-12-12T09:51:05.451 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2023-12-12T09:51:05.451 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2023-12-12T09:51:05.451 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2023-12-12T09:51:05.452 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2023-12-12T09:51:05.452 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2023-12-12T09:51:05.452 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2023-12-12T09:51:05.452 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.452 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-12T09:51:05.452 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2023-12-12T09:51:05.453 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2023-12-12T09:51:05.453 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.453 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2023-12-12T09:51:05.453 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-12T09:51:05.453 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-12T09:51:05.453 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.454 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2023-12-12T09:51:05.454 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2023-12-12T09:51:05.454 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.454 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2023-12-12T09:51:05.454 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2023-12-12T09:51:05.454 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-12-12T09:51:05.455 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2023-12-12T09:51:05.455 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2023-12-12T09:51:05.455 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2023-12-12T09:51:05.455 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2023-12-12T09:51:05.455 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-12T09:51:05.455 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-12T09:51:05.456 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2023-12-12T09:51:05.456 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2023-12-12T09:51:05.456 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-12-12T09:51:05.456 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2023-12-12T09:51:05.456 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2023-12-12T09:51:05.456 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2023-12-12T09:51:05.457 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2023-12-12T09:51:05.457 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2023-12-12T09:51:05.457 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2023-12-12T09:51:05.457 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2023-12-12T09:51:05.457 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2023-12-12T09:51:05.457 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2023-12-12T09:51:05.458 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2023-12-12T09:51:05.458 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2023-12-12T09:51:05.458 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2023-12-12T09:51:05.458 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2023-12-12T09:51:05.458 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-12-12T09:51:05.458 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2023-12-12T09:51:05.458 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2023-12-12T09:51:05.459 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2023-12-12T09:51:05.459 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2023-12-12T09:51:05.459 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2023-12-12T09:51:05.459 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2023-12-12T09:51:05.459 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2023-12-12T09:51:05.479 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-12T09:51:05.479 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-12T09:51:05.623 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-12T09:51:05.623 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:05.623 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-12T09:51:05.675 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:05.676 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-12T09:51:05.727 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:51:05.727 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:05.727 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-12T09:51:05.727 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-12T09:51:05.868 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:05.868 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:05.968 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:51:05.968 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:05.968 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-12T09:51:05.968 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-12T09:51:06.128 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:06.128 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:06.226 INFO:tasks.ceph:Setting up mon nodes... 2023-12-12T09:51:06.226 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-12T09:51:06.227 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-12T09:51:06.422 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-12T09:51:06.432 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-12T09:51:06.632 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-12T09:51:06.642 INFO:tasks.ceph:Setting up mds nodes... 2023-12-12T09:51:06.643 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-12T09:51:06.834 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-12T09:51:06.843 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-12T09:51:06.940 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-12T09:51:07.114 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-12T09:51:07.123 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-12T09:51:07.213 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-12T09:51:07.399 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-12T09:51:07.406 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-12T09:51:07.502 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-12T09:51:07.685 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-12T09:51:07.694 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-12T09:51:07.789 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-12T09:51:07.790 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-12T09:51:07.921 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-12T09:51:07.969 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-12T09:51:08.094 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-12T09:51:08.146 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-12T09:51:08.272 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-12T09:51:08.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-12T09:51:08.457 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-12T09:51:08.507 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-12T09:51:08.508 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-12T09:51:08.508 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-12T09:51:08.610 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:08.611 INFO:tasks.ceph:role: osd.0 2023-12-12T09:51:08.611 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:51:08.611 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-12T09:51:08.715 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-12T09:51:08.715 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-12T09:51:08.715 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-12T09:51:08.715 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-12T09:51:08.715 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-12T09:51:08.716 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-12-12T09:51:08.716 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-12T09:51:08.716 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-12T09:51:08.716 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-12T09:51:08.716 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-12T09:51:09.224 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-12-12T09:51:09.231 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-12-12T09:51:09.231 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-12T09:51:09.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-12T09:51:09.537 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-12T09:51:09.632 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:09.632 INFO:tasks.ceph:role: osd.1 2023-12-12T09:51:09.633 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:51:09.633 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-12T09:51:09.732 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-12T09:51:10.227 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-12-12T09:51:10.233 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-12-12T09:51:10.233 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-12T09:51:10.372 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-12T09:51:10.461 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-12T09:51:10.553 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:10.553 INFO:tasks.ceph:role: osd.2 2023-12-12T09:51:10.553 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:51:10.554 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-12T09:51:10.655 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-12T09:51:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-12T09:51:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-12T09:51:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-12T09:51:10.655 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-12T09:51:10.655 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-12-12T09:51:10.655 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-12T09:51:10.656 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-12T09:51:10.656 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-12T09:51:10.656 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-12T09:51:11.171 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-12-12T09:51:11.177 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-12-12T09:51:11.177 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-12T09:51:11.374 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-12T09:51:11.484 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-12T09:51:11.573 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:11.573 INFO:tasks.ceph:role: osd.3 2023-12-12T09:51:11.573 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:51:11.573 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-12T09:51:11.671 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-12T09:51:11.672 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-12T09:51:12.130 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-12-12T09:51:12.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-12-12T09:51:12.136 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-12T09:51:12.283 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-12T09:51:12.371 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:12.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:12.503+0000 7f7440800500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-12T09:51:12.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:12.506+0000 7f7440800500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-12T09:51:12.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:12.506+0000 7f7440800500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-12T09:51:12.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:12.506+0000 7f7440800500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-12T09:51:12.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:12.506+0000 7f7440800500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-12T09:51:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:12.515+0000 7f7440800500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-12T09:51:15.479 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-12T09:51:15.580 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:15.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:15.720+0000 7fa4d681d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-12T09:51:15.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:15.721+0000 7fa4d681d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-12T09:51:15.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:15.721+0000 7fa4d681d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-12T09:51:15.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:15.721+0000 7fa4d681d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-12T09:51:15.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:15.721+0000 7fa4d681d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-12T09:51:15.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:15.723+0000 7fa4d681d500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-12T09:51:18.639 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-12T09:51:18.730 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:18.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:18.860+0000 7fe959a21500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-12T09:51:18.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:18.861+0000 7fe959a21500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-12T09:51:18.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:18.861+0000 7fe959a21500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-12T09:51:18.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:18.861+0000 7fe959a21500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-12T09:51:18.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:18.861+0000 7fe959a21500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-12T09:51:18.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:18.863+0000 7fe959a21500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-12T09:51:21.810 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-12T09:51:21.908 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:22.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:22.047+0000 7fd631838500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-12T09:51:22.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:22.047+0000 7fd631838500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-12T09:51:22.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:22.048+0000 7fd631838500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-12T09:51:22.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:22.048+0000 7fd631838500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-12T09:51:22.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:22.048+0000 7fd631838500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-12T09:51:22.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:22.050+0000 7fd631838500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-12T09:51:25.235 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-12T09:51:25.335 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-12T09:51:25.440 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:25.440 INFO:tasks.ceph:role: osd.4 2023-12-12T09:51:25.440 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:51:25.440 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-12T09:51:25.545 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-12T09:51:25.545 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-12T09:51:25.545 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-12T09:51:25.545 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-12T09:51:25.545 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-12T09:51:25.545 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2023-12-12T09:51:25.546 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-12T09:51:25.546 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-12T09:51:25.546 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-12T09:51:25.546 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-12T09:51:26.078 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2023-12-12T09:51:26.084 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2023-12-12T09:51:26.085 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-12T09:51:26.233 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-12T09:51:26.331 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-12T09:51:26.423 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:26.423 INFO:tasks.ceph:role: osd.5 2023-12-12T09:51:26.423 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:51:26.423 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-12T09:51:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-12T09:51:27.056 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2023-12-12T09:51:27.062 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2023-12-12T09:51:27.063 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-12T09:51:27.225 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-12T09:51:27.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-12T09:51:27.411 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:27.411 INFO:tasks.ceph:role: osd.6 2023-12-12T09:51:27.411 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:51:27.411 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-12T09:51:27.511 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-12T09:51:27.511 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-12T09:51:27.511 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-12T09:51:27.511 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-12T09:51:27.512 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-12T09:51:27.512 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2023-12-12T09:51:27.512 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-12T09:51:27.512 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-12T09:51:27.512 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-12T09:51:27.512 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-12T09:51:28.034 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2023-12-12T09:51:28.041 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2023-12-12T09:51:28.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-12T09:51:28.196 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-12T09:51:28.295 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-12T09:51:28.381 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-12T09:51:28.381 INFO:tasks.ceph:role: osd.7 2023-12-12T09:51:28.381 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:51:28.381 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-12T09:51:28.481 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-12T09:51:28.481 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-12T09:51:28.481 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-12T09:51:29.020 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2023-12-12T09:51:29.026 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2023-12-12T09:51:29.027 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-12T09:51:29.182 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-12T09:51:29.286 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:29.439 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:29.438+0000 7fae8ccaf500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-12T09:51:29.440 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:29.439+0000 7fae8ccaf500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-12T09:51:29.440 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:29.439+0000 7fae8ccaf500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-12T09:51:29.440 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:29.439+0000 7fae8ccaf500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-12T09:51:29.440 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:29.439+0000 7fae8ccaf500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-12T09:51:29.443 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:29.442+0000 7fae8ccaf500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-12T09:51:32.382 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-12T09:51:32.484 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:32.542 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:51:32.551 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:51:32.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:32.622+0000 7f1210d4a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-12T09:51:32.624 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:32.623+0000 7f1210d4a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-12T09:51:32.624 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:32.623+0000 7f1210d4a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-12T09:51:32.624 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:32.623+0000 7f1210d4a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-12T09:51:32.624 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:32.623+0000 7f1210d4a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-12T09:51:32.626 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:32.625+0000 7f1210d4a500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-12T09:51:35.553 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-12T09:51:35.649 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:35.789 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:35.788+0000 7fd285c12500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-12T09:51:35.790 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:35.789+0000 7fd285c12500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-12T09:51:35.790 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:35.789+0000 7fd285c12500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-12T09:51:35.790 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:35.789+0000 7fd285c12500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-12T09:51:35.790 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:35.789+0000 7fd285c12500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-12T09:51:35.792 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:35.791+0000 7fd285c12500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-12T09:51:38.974 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-12T09:51:39.074 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:39.213 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:39.212+0000 7f964178b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-12T09:51:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:39.213+0000 7f964178b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-12T09:51:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:39.213+0000 7f964178b500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-12T09:51:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:39.213+0000 7f964178b500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-12T09:51:39.214 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:39.213+0000 7f964178b500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-12T09:51:39.216 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:39.215+0000 7f964178b500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-12T09:51:42.398 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-12T09:51:42.499 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-12T09:51:42.499 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:42.499 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-12T09:51:42.604 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:42.604 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-12T09:51:42.700 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:42.700 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-12T09:51:42.796 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:42.797 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-12T09:51:42.950 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:42.950 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-12T09:51:43.055 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:43.055 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-12T09:51:43.159 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:43.159 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-12T09:51:43.264 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:43.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-12T09:51:43.369 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:43.369 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-12T09:51:43.473 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:43.473 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-12T09:51:43.570 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:43.571 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-12T09:51:43.666 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:43.666 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-12T09:51:43.759 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:43.759 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-12T09:51:43.859 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:43.859 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-12T09:51:43.955 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:43.956 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-12T09:51:44.021 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:44.021 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-12T09:51:44.083 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:44.083 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-12T09:51:44.149 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:44.149 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-12T09:51:44.213 INFO:tasks.ceph:Adding keys to all mons... 2023-12-12T09:51:44.214 DEBUG:teuthology.orchestra.run.smithi072:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-12T09:51:44.223 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-12T09:51:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2023-12-12T09:51:44.311 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAKLXhlmRkdGRAAV7LEG0mSAIVyOfeB2+L1fQ== 2023-12-12T09:51:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2023-12-12T09:51:44.311 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAKLXhlvoC3MRAAoAmTetbcBYxKsaGaD6tAIg== 2023-12-12T09:51:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2023-12-12T09:51:44.311 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQALLXhlkWvDBhAAKV9ZAou23rMrzJJ4GURxGw== 2023-12-12T09:51:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2023-12-12T09:51:44.312 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAQLXhldS0YHhAANAsQberqgVrbCB5TcgrmsA== 2023-12-12T09:51:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2023-12-12T09:51:44.312 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQATLXhlyVgFKxAA3xZtCVmkuMrqrZCWloqoMA== 2023-12-12T09:51:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2023-12-12T09:51:44.312 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAWLXhlqrpdMxAAmfxGvPquBX/kyiGCi/rjQg== 2023-12-12T09:51:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2023-12-12T09:51:44.313 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAaLXhl5nXjAhAACmfr78GlROpqJ7UKUwer7A== 2023-12-12T09:51:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2023-12-12T09:51:44.313 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAKLXhlujKiJRAACswJ4xyotHStzLDuXHlGPA== 2023-12-12T09:51:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2023-12-12T09:51:44.313 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQALLXhlEGCfFxAAmpAYcdOKIMI4Wg2GyD2V1w== 2023-12-12T09:51:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2023-12-12T09:51:44.314 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQALLXhlJcbVKBAAPs/uN737wS83OIndY0JCZQ== 2023-12-12T09:51:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2023-12-12T09:51:44.314 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAhLXhlLaouGhAAqURr0qTsHKQby0Th8coLCA== 2023-12-12T09:51:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2023-12-12T09:51:44.315 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAkLXhlGVwmJRAAX0ujEA/JpOrqD9N9mRR1hg== 2023-12-12T09:51:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2023-12-12T09:51:44.315 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAnLXhl5VkLLxAAY8lCytX5JEfHFPZ3ABewnw== 2023-12-12T09:51:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2023-12-12T09:51:44.315 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQArLXhlLJaxDBAAGd8VZfX+Ofg80KrvflSEcA== 2023-12-12T09:51:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2023-12-12T09:51:44.316 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQALLXhlQjDrNhAAcu4UiqhTEuide2bwToXy+Q== 2023-12-12T09:51:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:[client.2] 2023-12-12T09:51:44.316 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAMLXhlPlybBRAAVk0eZQW2HZk9y6TwvEWIkQ== 2023-12-12T09:51:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:[client.1] 2023-12-12T09:51:44.317 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAMLXhlMLcyEBAAihZy2JhlSzOc1JbYT37pIw== 2023-12-12T09:51:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:[client.3] 2023-12-12T09:51:44.317 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAMLXhlEkM2GxAAvsUAfS00lFnfoviP3gE/mg== 2023-12-12T09:51:44.318 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.y] 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAKLXhlmRkdGRAAV7LEG0mSAIVyOfeB2+L1fQ== 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a] 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAKLXhlvoC3MRAAoAmTetbcBYxKsaGaD6tAIg== 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c] 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQALLXhlkWvDBhAAKV9ZAou23rMrzJJ4GURxGw== 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.0] 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAQLXhldS0YHhAANAsQberqgVrbCB5TcgrmsA== 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.1] 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQATLXhlyVgFKxAA3xZtCVmkuMrqrZCWloqoMA== 2023-12-12T09:51:44.319 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.2] 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAWLXhlqrpdMxAAmfxGvPquBX/kyiGCi/rjQg== 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.3] 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAaLXhl5nXjAhAACmfr78GlROpqJ7UKUwer7A== 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.x] 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAKLXhlujKiJRAACswJ4xyotHStzLDuXHlGPA== 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b] 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQALLXhlEGCfFxAAmpAYcdOKIMI4Wg2GyD2V1w== 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d] 2023-12-12T09:51:44.320 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQALLXhlJcbVKBAAPs/uN737wS83OIndY0JCZQ== 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.4] 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAhLXhlLaouGhAAqURr0qTsHKQby0Th8coLCA== 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.5] 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAkLXhlGVwmJRAAX0ujEA/JpOrqD9N9mRR1hg== 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.6] 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAnLXhl5VkLLxAAY8lCytX5JEfHFPZ3ABewnw== 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.7] 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQArLXhlLJaxDBAAGd8VZfX+Ofg80KrvflSEcA== 2023-12-12T09:51:44.321 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2023-12-12T09:51:44.322 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQALLXhlQjDrNhAAcu4UiqhTEuide2bwToXy+Q== 2023-12-12T09:51:44.322 INFO:teuthology.orchestra.run.smithi072.stdout:[client.2] 2023-12-12T09:51:44.322 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAMLXhlPlybBRAAVk0eZQW2HZk9y6TwvEWIkQ== 2023-12-12T09:51:44.322 INFO:teuthology.orchestra.run.smithi072.stdout:[client.1] 2023-12-12T09:51:44.322 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAMLXhlMLcyEBAAihZy2JhlSzOc1JbYT37pIw== 2023-12-12T09:51:44.322 INFO:teuthology.orchestra.run.smithi072.stdout:[client.3] 2023-12-12T09:51:44.322 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAMLXhlEkM2GxAAvsUAfS00lFnfoviP3gE/mg== 2023-12-12T09:51:44.324 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-12T09:51:44.333 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-12T09:51:44.480 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-12T09:51:44.486 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-12T09:51:44.637 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-12T09:51:44.647 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-12T09:51:44.791 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:44.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:44.950 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:44.959 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:45.096 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:45.104 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:45.245 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:45.250 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:45.396 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-12T09:51:45.405 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-12T09:51:45.548 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-12T09:51:45.557 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-12T09:51:45.700 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-12T09:51:45.706 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-12T09:51:45.848 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:45.854 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:46.001 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:46.010 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:46.162 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:46.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:46.316 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:46.324 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-12T09:51:46.472 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-12T09:51:46.477 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-12T09:51:46.630 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-12T09:51:46.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-12T09:51:46.788 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-12T09:51:46.793 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-12T09:51:46.937 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-12T09:51:46.942 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-12T09:51:47.082 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-12T09:51:47.082 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-12T09:51:47.177 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-12T09:51:47.326 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-12T09:51:47.416 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-12T09:51:47.517 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-12T09:51:47.676 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-12T09:51:47.773 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-12T09:51:47.870 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-12T09:51:48.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-12T09:51:48.119 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:48.128 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-12T09:51:48.180 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-12T09:51:48.181 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-12T09:51:48.181 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-12T09:51:48.187 INFO:tasks.ceph.mon.a:Started 2023-12-12T09:51:48.187 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-12T09:51:48.187 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-12T09:51:48.193 INFO:tasks.ceph.mon.b:Started 2023-12-12T09:51:48.193 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-12T09:51:48.193 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-12T09:51:48.198 INFO:tasks.ceph.mon.c:Started 2023-12-12T09:51:48.199 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-12T09:51:48.199 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-12T09:51:48.199 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-12T09:51:48.207 INFO:tasks.ceph.mgr.y:Started 2023-12-12T09:51:48.207 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-12T09:51:48.208 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-12T09:51:48.213 INFO:tasks.ceph.mgr.x:Started 2023-12-12T09:51:48.213 INFO:tasks.ceph:Setting crush tunables to default 2023-12-12T09:51:48.213 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd crush tunables default 2023-12-12T09:51:48.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.393+0000 7f569a553640 1 Processor -- start 2023-12-12T09:51:48.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.393+0000 7f569a553640 1 -- start start 2023-12-12T09:51:48.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.394+0000 7f569a553640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5694160b40 0x7f56940b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:48.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.394+0000 7f569a553640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56940d0af0 con 0x7f56940b2a50 2023-12-12T09:51:48.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.394+0000 7f569a553640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5694160f20 con 0x7f56940b35d0 2023-12-12T09:51:48.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.394+0000 7f569a553640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56940d1520 con 0x7f5694160b40 2023-12-12T09:51:48.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.395+0000 7f5693fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5694160b40 0x7f56940b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:48.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.395+0000 7f568bfff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f56940b2a50 0x7f56940b2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43576/0 (socket says 172.21.15.72:43576) 2023-12-12T09:51:48.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.395+0000 7f568bfff640 1 -- 172.21.15.72:0/530301244 learned_addr learned my addr 172.21.15.72:0/530301244 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:51:48.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.395+0000 7f5693fff640 1 -- 172.21.15.72:0/530301244 >> v2:172.21.15.113:3300/0 conn(0x7f5694160b40 msgr2=0x7f56940b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-12T09:51:48.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.395+0000 7f5693fff640 1 -- 172.21.15.72:0/530301244 >> v2:172.21.15.113:3300/0 conn(0x7f5694160b40 msgr2=0x7f56940b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-12T09:51:48.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.395+0000 7f5693fff640 1 --2- 172.21.15.72:0/530301244 >> v2:172.21.15.113:3300/0 conn(0x7f5694160b40 0x7f56940b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-12T09:51:48.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.395+0000 7f5693fff640 1 --2- 172.21.15.72:0/530301244 >> v2:172.21.15.113:3300/0 conn(0x7f5694160b40 0x7f56940b46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:51:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840028889 0 0) 0x7f56940d0af0 con 0x7f56940b2a50 2023-12-12T09:51:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5680003520 con 0x7f56940b2a50 2023-12-12T09:51:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127648207 0 0) 0x7f5694160f20 con 0x7f56940b35d0 2023-12-12T09:51:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56940d0af0 con 0x7f56940b35d0 2023-12-12T09:51:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 522783530 0 0) 0x7f5680003520 con 0x7f56940b2a50 2023-12-12T09:51:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 >> v2:172.21.15.113:3300/0 conn(0x7f5694160b40 msgr2=0x7f56940b46a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-12T09:51:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 --2- 172.21.15.72:0/530301244 >> v2:172.21.15.113:3300/0 conn(0x7f5694160b40 0x7f56940b46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 >> v1:172.21.15.113:6789/0 conn(0x7f56940b35d0 legacy=0x7f56940cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56940cf040 con 0x7f56940b2a50 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f567c001670 con 0x7f56940b2a50 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 >> v1:172.21.15.72:6789/0 conn(0x7f56940b2a50 legacy=0x7f56940b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680003b00 0x7f5680003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56800047c0 0x7f5680004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5680005180 0x7f568000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56940d1520 con 0x7f5680003b00 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568000c0b0 con 0x7f5680005180 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568000c270 con 0x7f56800047c0 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.413+0000 7f5698af0640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5680005180 0x7f568000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.414+0000 7f568bfff640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56800047c0 0x7f5680004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.414+0000 7f5693fff640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680003b00 0x7f5680003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.414+0000 7f5698af0640 1 -- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56800047c0 msgr2=0x7f5680004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:48.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.414+0000 7f5698af0640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56800047c0 0x7f5680004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.414+0000 7f5698af0640 1 -- 172.21.15.72:0/530301244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680003b00 msgr2=0x7f5680003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:48.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.414+0000 7f5698af0640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680003b00 0x7f5680003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.414+0000 7f5698af0640 1 -- 172.21.15.72:0/530301244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5680003520 con 0x7f5680005180 2023-12-12T09:51:48.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.414+0000 7f568bfff640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56800047c0 0x7f5680004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:51:48.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.415+0000 7f5698af0640 1 -- 172.21.15.72:0/530301244 reap_dead start 2023-12-12T09:51:48.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.415+0000 7f5698af0640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5680005180 0x7f568000b980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f568c001fa0 tx=0x7f56800057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eaa6437ee8bfe12 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:48.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c003b70 con 0x7f5680005180 2023-12-12T09:51:48.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c003cd0 con 0x7f5680005180 2023-12-12T09:51:48.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f56937fe640 1 -- 172.21.15.72:0/530301244 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c005310 con 0x7f5680005180 2023-12-12T09:51:48.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f569a553640 1 -- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5680005180 msgr2=0x7f568000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:48.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f569a553640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5680005180 0x7f568000b980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f568c001fa0 tx=0x7f56800057f0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f569a553640 1 -- 172.21.15.72:0/530301244 shutdown_connections 2023-12-12T09:51:48.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f569a553640 1 --2- 172.21.15.72:0/530301244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5680005180 0x7f568000b980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f569a553640 1 -- 172.21.15.72:0/530301244 >> 172.21.15.72:0/530301244 conn(0x7f569409c420 msgr2=0x7f56940d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:48.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f569a553640 1 -- 172.21.15.72:0/530301244 shutdown_connections 2023-12-12T09:51:48.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.422+0000 7f569a553640 1 -- 172.21.15.72:0/530301244 wait complete. 2023-12-12T09:51:48.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.424+0000 7f569a553640 1 Processor -- start 2023-12-12T09:51:48.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.424+0000 7f569a553640 1 -- start start 2023-12-12T09:51:48.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f569a553640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680005180 0x7f56940b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f569a553640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56940b8aa0 0x7f569414e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f569a553640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f569415ec80 0x7f569415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f569a553640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56940b6ee0 con 0x7f5680005180 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f569a553640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569415f640 con 0x7f56940b8aa0 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f569a553640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56940cb630 con 0x7f569415ec80 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f5693fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680005180 0x7f56940b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f5693fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680005180 0x7f56940b69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41882/0 (socket says 172.21.15.72:41882) 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.425+0000 7f5693fff640 1 -- 172.21.15.72:0/3816064934 learned_addr learned my addr 172.21.15.72:0/3816064934 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f568bfff640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56940b8aa0 0x7f569414e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f5698af0640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f569415ec80 0x7f569415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f5693fff640 1 -- 172.21.15.72:0/3816064934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f569415ec80 msgr2=0x7f569415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f5693fff640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f569415ec80 0x7f569415f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f5693fff640 1 -- 172.21.15.72:0/3816064934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56940b8aa0 msgr2=0x7f569414e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f5693fff640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56940b8aa0 0x7f569414e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f5693fff640 1 -- 172.21.15.72:0/3816064934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56940b70d0 con 0x7f5680005180 2023-12-12T09:51:48.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f5693fff640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680005180 0x7f56940b69a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f56840108c0 tx=0x7f5684010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:48.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f56917fa640 1 -- 172.21.15.72:0/3816064934 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684010eb0 con 0x7f5680005180 2023-12-12T09:51:48.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f56917fa640 1 -- 172.21.15.72:0/3816064934 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568400e9c0 con 0x7f5680005180 2023-12-12T09:51:48.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f56917fa640 1 -- 172.21.15.72:0/3816064934 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400ed70 con 0x7f5680005180 2023-12-12T09:51:48.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.426+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56940b73b0 con 0x7f5680005180 2023-12-12T09:51:48.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.427+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56940cbb60 con 0x7f5680005180 2023-12-12T09:51:48.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.427+0000 7f56917fa640 1 -- 172.21.15.72:0/3816064934 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5684016780 con 0x7f5680005180 2023-12-12T09:51:48.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.427+0000 7f56917fa640 1 -- 172.21.15.72:0/3816064934 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5684014860 con 0x7f5680005180 2023-12-12T09:51:48.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.428+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56940cbcf0 con 0x7f5680005180 2023-12-12T09:51:48.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.431+0000 7f56917fa640 1 -- 172.21.15.72:0/3816064934 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f568403ece0 con 0x7f5680005180 2023-12-12T09:51:48.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.503+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f56940d1310 con 0x7f5680005180 2023-12-12T09:51:48.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.509+0000 7f56917fa640 1 -- 172.21.15.72:0/3816064934 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f568402d020 con 0x7f5680005180 2023-12-12T09:51:48.511 INFO:teuthology.orchestra.run.smithi072.stderr:adjusted tunables profile to default 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680005180 msgr2=0x7f56940b69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680005180 0x7f56940b69a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f56840108c0 tx=0x7f5684010cc0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 shutdown_connections 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f569415ec80 0x7f569415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56940b8aa0 0x7f569414e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 --2- 172.21.15.72:0/3816064934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5680005180 0x7f56940b69a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 >> 172.21.15.72:0/3816064934 conn(0x7f569409c420 msgr2=0x7f56940b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 shutdown_connections 2023-12-12T09:51:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:48.510+0000 7f569a553640 1 -- 172.21.15.72:0/3816064934 wait complete. 2023-12-12T09:51:48.530 INFO:tasks.ceph:check_enable_crimson: False 2023-12-12T09:51:48.530 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-12T09:51:48.530 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:48.530 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-12T09:51:48.609 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:48.609 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-12T09:51:48.733 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:48.733 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-12T09:51:48.861 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:51:48.861 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-12T09:51:48.996 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:48.996 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-12T09:51:49.086 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:49.086 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-12T09:51:49.114 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:49.111+0000 7fa3ce413040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:51:49.165 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:49.164+0000 7f7236412040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:51:49.175 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:49.173+0000 7fa3ce413040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:51:49.202 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:49.202 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-12T09:51:49.229 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:49.229+0000 7f7236412040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:51:49.287 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:51:49.287 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-12T09:51:49.415 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new be73d3b6-8cce-44e5-b7e7-662e308d68e8 0 2023-12-12T09:51:49.651 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.649+0000 7f6ae13d4640 1 Processor -- start 2023-12-12T09:51:49.652 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.649+0000 7f6ae13d4640 1 -- start start 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.651+0000 7f6ae13d4640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6adc0b3350 0x7f6adc0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.651+0000 7f6ae13d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6adc0cfe70 0x7f6adc0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.651+0000 7f6ae13d4640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc0c6b80 con 0x7f6adc0b3730 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.651+0000 7f6ae13d4640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc0d3bf0 con 0x7f6adc0b3350 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.651+0000 7f6ae13d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc0d3ed0 con 0x7f6adc0cfe70 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.651+0000 7f6ae0bd3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6adc0cfe70 0x7f6adc0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.651+0000 7f6ae0bd3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6adc0cfe70 0x7f6adc0cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54422/0 (socket says 172.21.15.113:54422) 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.651+0000 7f6ae0bd3640 1 -- 172.21.15.113:0/401401866 learned_addr learned my addr 172.21.15.113:0/401401866 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.652+0000 7f6adbfff640 1 --2- 172.21.15.113:0/401401866 >> v2:172.21.15.72:3300/0 conn(0x7f6adc0b3350 0x7f6adc0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.652+0000 7f6ae0bd3640 1 -- 172.21.15.113:0/401401866 >> v2:172.21.15.72:3300/0 conn(0x7f6adc0b3350 msgr2=0x7f6adc0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:49.654 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.652+0000 7f6ae0bd3640 1 --2- 172.21.15.113:0/401401866 >> v2:172.21.15.72:3300/0 conn(0x7f6adc0b3350 0x7f6adc0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.655 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.652+0000 7f6ae0bd3640 1 -- 172.21.15.113:0/401401866 >> v1:172.21.15.72:6789/0 conn(0x7f6adc0b3730 legacy=0x7f6adc0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:49.655 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.652+0000 7f6ae0bd3640 1 -- 172.21.15.113:0/401401866 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0d18c0 con 0x7f6adc0cfe70 2023-12-12T09:51:49.655 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.653+0000 7f6ae0bd3640 1 --2- 172.21.15.113:0/401401866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 0x7f6adc0cdef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6adc1622a0 tx=0x7f6ad402bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c912e36ec9daa82e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:49.655 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.653+0000 7f6adaffd640 1 -- 172.21.15.113:0/401401866 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad402ca80 con 0x7f6adc0cfe70 2023-12-12T09:51:49.655 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.653+0000 7f6adaffd640 1 -- 172.21.15.113:0/401401866 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad402cbe0 con 0x7f6adc0cfe70 2023-12-12T09:51:49.655 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.653+0000 7f6adaffd640 1 -- 172.21.15.113:0/401401866 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40343f0 con 0x7f6adc0cfe70 2023-12-12T09:51:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.654+0000 7f6ae13d4640 1 -- 172.21.15.113:0/401401866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 msgr2=0x7f6adc0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.654+0000 7f6ae13d4640 1 --2- 172.21.15.113:0/401401866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 0x7f6adc0cdef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6adc1622a0 tx=0x7f6ad402bee0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.654+0000 7f6ae13d4640 1 -- 172.21.15.113:0/401401866 shutdown_connections 2023-12-12T09:51:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.654+0000 7f6ae13d4640 1 --2- 172.21.15.113:0/401401866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 0x7f6adc0cdef0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.656 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.654+0000 7f6ae13d4640 1 --2- 172.21.15.113:0/401401866 >> v2:172.21.15.72:3300/0 conn(0x7f6adc0b3350 0x7f6adc0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.657 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.654+0000 7f6ae13d4640 1 -- 172.21.15.113:0/401401866 >> 172.21.15.113:0/401401866 conn(0x7f6adc0d3030 msgr2=0x7f6adc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:49.657 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.654+0000 7f6ae13d4640 1 -- 172.21.15.113:0/401401866 shutdown_connections 2023-12-12T09:51:49.657 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.654+0000 7f6ae13d4640 1 -- 172.21.15.113:0/401401866 wait complete. 2023-12-12T09:51:49.658 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.656+0000 7f6ae13d4640 1 Processor -- start 2023-12-12T09:51:49.658 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.656+0000 7f6ae13d4640 1 -- start start 2023-12-12T09:51:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae13d4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6adc0b3350 0x7f6adc153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae13d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6adc0b3730 0x7f6adc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae13d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 0x7f6adc15dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae13d4640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc15eb50 con 0x7f6adc0b3350 2023-12-12T09:51:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae13d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0ccff0 con 0x7f6adc0cfe70 2023-12-12T09:51:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae13d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc153ba0 con 0x7f6adc0b3730 2023-12-12T09:51:49.662 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6adb7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6adc0b3730 0x7f6adc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.663 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae0bd3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 0x7f6adc15dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.663 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6adb7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6adc0b3730 0x7f6adc15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:39952/0 (socket says 172.21.15.113:39952) 2023-12-12T09:51:49.663 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6adb7fe640 1 -- 172.21.15.113:0/2524099155 learned_addr learned my addr 172.21.15.113:0/2524099155 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:49.663 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6adbfff640 1 --2- 172.21.15.113:0/2524099155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6adc0b3350 0x7f6adc153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.663 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae0bd3640 1 -- 172.21.15.113:0/2524099155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6adc0b3730 msgr2=0x7f6adc15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:49.663 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae0bd3640 1 --2- 172.21.15.113:0/2524099155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6adc0b3730 0x7f6adc15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.663 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae0bd3640 1 -- 172.21.15.113:0/2524099155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6adc0b3350 msgr2=0x7f6adc153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:49.663 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae0bd3640 1 --2- 172.21.15.113:0/2524099155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6adc0b3350 0x7f6adc153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.664 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.657+0000 7f6ae0bd3640 1 -- 172.21.15.113:0/2524099155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4009b30 con 0x7f6adc0cfe70 2023-12-12T09:51:49.664 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.658+0000 7f6ae0bd3640 1 --2- 172.21.15.113:0/2524099155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 0x7f6adc15dbd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4032000 tx=0x7f6ad4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:49.664 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.658+0000 7f6ad8ff9640 1 -- 172.21.15.113:0/2524099155 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4009510 con 0x7f6adc0cfe70 2023-12-12T09:51:49.664 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.658+0000 7f6ad8ff9640 1 -- 172.21.15.113:0/2524099155 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad403e450 con 0x7f6adc0cfe70 2023-12-12T09:51:49.664 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.658+0000 7f6ad8ff9640 1 -- 172.21.15.113:0/2524099155 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad403e800 con 0x7f6adc0cfe70 2023-12-12T09:51:49.664 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.658+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc0b48a0 con 0x7f6adc0cfe70 2023-12-12T09:51:49.664 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.658+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc160370 con 0x7f6adc0cfe70 2023-12-12T09:51:49.664 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.658+0000 7f6ad8ff9640 1 -- 172.21.15.113:0/2524099155 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6ad402cd50 con 0x7f6adc0cfe70 2023-12-12T09:51:49.665 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.658+0000 7f6ad8ff9640 1 -- 172.21.15.113:0/2524099155 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6ad4048810 con 0x7f6adc0cfe70 2023-12-12T09:51:49.665 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.659+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc066d20 con 0x7f6adc0cfe70 2023-12-12T09:51:49.666 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.664+0000 7f6ad8ff9640 1 -- 172.21.15.113:0/2524099155 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6ad4043020 con 0x7f6adc0cfe70 2023-12-12T09:51:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.742+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "be73d3b6-8cce-44e5-b7e7-662e308d68e8", "id": 0} v 0) v1 -- 0x7f6adc074450 con 0x7f6adc0cfe70 2023-12-12T09:51:49.750 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.748+0000 7f6ad8ff9640 1 -- 172.21.15.113:0/2524099155 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "be73d3b6-8cce-44e5-b7e7-662e308d68e8", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6ad403ee40 con 0x7f6adc0cfe70 2023-12-12T09:51:49.750 INFO:teuthology.orchestra.run.smithi113.stdout:0 2023-12-12T09:51:49.751 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 msgr2=0x7f6adc15dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:49.751 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 --2- 172.21.15.113:0/2524099155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 0x7f6adc15dbd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4032000 tx=0x7f6ad4004a40 comp rx=0 tx=0).stop 2023-12-12T09:51:49.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 shutdown_connections 2023-12-12T09:51:49.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 --2- 172.21.15.113:0/2524099155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6adc0cfe70 0x7f6adc15dbd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 --2- 172.21.15.113:0/2524099155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6adc0b3730 0x7f6adc15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 --2- 172.21.15.113:0/2524099155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6adc0b3350 0x7f6adc153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 >> 172.21.15.113:0/2524099155 conn(0x7f6adc0d3030 msgr2=0x7f6adc0b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:49.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 shutdown_connections 2023-12-12T09:51:49.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.749+0000 7f6ae13d4640 1 -- 172.21.15.113:0/2524099155 wait complete. 2023-12-12T09:51:49.769 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new ac551d10-c0fd-4277-ae22-e8044ec35a11 1 2023-12-12T09:51:49.816 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:49.814+0000 7fa3ce413040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:51:49.881 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:49.879+0000 7fa3ce413040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:51:49.885 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:49.884+0000 7f7236412040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:51:49.935 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:49.932+0000 7fa3ce413040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:51:49.949 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:49.947+0000 7f7236412040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:51:49.954 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.951+0000 7f632094e640 1 Processor -- start 2023-12-12T09:51:49.954 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.952+0000 7f632094e640 1 -- start start 2023-12-12T09:51:49.958 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f632094e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.958 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f632094e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f631c0b3730 0x7f631c0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.959 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f632094e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631c0c6b80 con 0x7f631c0b40e0 2023-12-12T09:51:49.959 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f632094e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c0d3bf0 con 0x7f631c0b3730 2023-12-12T09:51:49.959 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f632094e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0d3ed0 con 0x7f631c0b3350 2023-12-12T09:51:49.959 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.960 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f6313fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f631c0b3730 0x7f631c0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.960 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54530/0 (socket says 172.21.15.113:54530) 2023-12-12T09:51:49.960 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.953+0000 7f631b7fe640 1 -- 172.21.15.113:0/1703527507 learned_addr learned my addr 172.21.15.113:0/1703527507 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:49.960 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.954+0000 7f631b7fe640 1 -- 172.21.15.113:0/1703527507 >> v2:172.21.15.113:3300/0 conn(0x7f631c0b3730 msgr2=0x7f631c0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:49.960 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.954+0000 7f631b7fe640 1 --2- 172.21.15.113:0/1703527507 >> v2:172.21.15.113:3300/0 conn(0x7f631c0b3730 0x7f631c0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.961 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.954+0000 7f631b7fe640 1 -- 172.21.15.113:0/1703527507 >> v1:172.21.15.72:6789/0 conn(0x7f631c0b40e0 legacy=0x7f631c161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:49.961 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.954+0000 7f631b7fe640 1 -- 172.21.15.113:0/1703527507 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0d18c0 con 0x7f631c0b3350 2023-12-12T09:51:49.961 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.955+0000 7f631b7fe640 1 --2- 172.21.15.113:0/1703527507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b3ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f630c009870 tx=0x7f630c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f11b8f404c493399 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:49.961 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.955+0000 7f631affd640 1 -- 172.21.15.113:0/1703527507 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c031640 con 0x7f631c0b3350 2023-12-12T09:51:49.961 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.955+0000 7f631affd640 1 -- 172.21.15.113:0/1703527507 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c0317a0 con 0x7f631c0b3350 2023-12-12T09:51:49.961 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.955+0000 7f631affd640 1 -- 172.21.15.113:0/1703527507 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c031b30 con 0x7f631c0b3350 2023-12-12T09:51:49.961 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.956+0000 7f632094e640 1 -- 172.21.15.113:0/1703527507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 msgr2=0x7f631c0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:49.962 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.956+0000 7f632094e640 1 --2- 172.21.15.113:0/1703527507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b3ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f630c009870 tx=0x7f630c02b8f0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.962 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.956+0000 7f632094e640 1 -- 172.21.15.113:0/1703527507 shutdown_connections 2023-12-12T09:51:49.962 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.956+0000 7f632094e640 1 --2- 172.21.15.113:0/1703527507 >> v2:172.21.15.113:3300/0 conn(0x7f631c0b3730 0x7f631c0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.962 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.956+0000 7f632094e640 1 --2- 172.21.15.113:0/1703527507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b3ba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.962 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.956+0000 7f632094e640 1 -- 172.21.15.113:0/1703527507 >> 172.21.15.113:0/1703527507 conn(0x7f631c0d3030 msgr2=0x7f631c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:49.962 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.956+0000 7f632094e640 1 -- 172.21.15.113:0/1703527507 shutdown_connections 2023-12-12T09:51:49.963 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.956+0000 7f632094e640 1 -- 172.21.15.113:0/1703527507 wait complete. 2023-12-12T09:51:49.967 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.957+0000 7f632094e640 1 Processor -- start 2023-12-12T09:51:49.967 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.958+0000 7f632094e640 1 -- start start 2023-12-12T09:51:49.967 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.958+0000 7f632094e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.967 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.958+0000 7f632094e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f631c0b3730 0x7f631c0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.968 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f632094e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f631c0b40e0 0x7f631c1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:49.968 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f632094e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0cd0f0 con 0x7f631c0b40e0 2023-12-12T09:51:49.968 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f632094e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c1f2fa0 con 0x7f631c0b3730 2023-12-12T09:51:49.968 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f632094e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0b83c0 con 0x7f631c0b3350 2023-12-12T09:51:49.968 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.968 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54532/0 (socket says 172.21.15.113:54532) 2023-12-12T09:51:49.968 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f6313fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f631c0b3730 0x7f631c0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.969 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631b7fe640 1 -- 172.21.15.113:0/464634316 learned_addr learned my addr 172.21.15.113:0/464634316 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:49.969 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631bfff640 1 --2- 172.21.15.113:0/464634316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f631c0b40e0 0x7f631c1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:49.969 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631b7fe640 1 -- 172.21.15.113:0/464634316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f631c0b3730 msgr2=0x7f631c0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:49.969 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631b7fe640 1 --2- 172.21.15.113:0/464634316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f631c0b3730 0x7f631c0ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.969 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631b7fe640 1 -- 172.21.15.113:0/464634316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f631c0b40e0 msgr2=0x7f631c1550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:49.969 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631b7fe640 1 --2- 172.21.15.113:0/464634316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f631c0b40e0 0x7f631c1550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:49.970 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.959+0000 7f631b7fe640 1 -- 172.21.15.113:0/464634316 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c0094d0 con 0x7f631c0b3350 2023-12-12T09:51:49.970 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.960+0000 7f631b7fe640 1 --2- 172.21.15.113:0/464634316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b7420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f630c000c00 tx=0x7f630c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:49.970 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.960+0000 7f6318ff9640 1 -- 172.21.15.113:0/464634316 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c03a930 con 0x7f631c0b3350 2023-12-12T09:51:49.970 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.960+0000 7f6318ff9640 1 -- 172.21.15.113:0/464634316 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c03aa90 con 0x7f631c0b3350 2023-12-12T09:51:49.970 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.960+0000 7f6318ff9640 1 -- 172.21.15.113:0/464634316 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c03ae40 con 0x7f631c0b3350 2023-12-12T09:51:49.970 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.960+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c0b48a0 con 0x7f631c0b3350 2023-12-12T09:51:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.960+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c15c970 con 0x7f631c0b3350 2023-12-12T09:51:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.960+0000 7f6318ff9640 1 -- 172.21.15.113:0/464634316 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f630c02f050 con 0x7f631c0b3350 2023-12-12T09:51:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.960+0000 7f6318ff9640 1 -- 172.21.15.113:0/464634316 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f630c044930 con 0x7f631c0b3350 2023-12-12T09:51:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.961+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c074450 con 0x7f631c0b3350 2023-12-12T09:51:49.971 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:49.964+0000 7f6318ff9640 1 -- 172.21.15.113:0/464634316 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f630c035020 con 0x7f631c0b3350 2023-12-12T09:51:50.001 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:49.999+0000 7f7236412040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:51:50.001 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:49.999+0000 7fa3ce413040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:51:50.044 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.041+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac551d10-c0fd-4277-ae22-e8044ec35a11", "id": 1} v 0) v1 -- 0x7f631c072830 con 0x7f631c0b3350 2023-12-12T09:51:50.052 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.050+0000 7f6318ff9640 1 -- 172.21.15.113:0/464634316 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac551d10-c0fd-4277-ae22-e8044ec35a11", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f630c02d040 con 0x7f631c0b3350 2023-12-12T09:51:50.053 INFO:teuthology.orchestra.run.smithi113.stdout:1 2023-12-12T09:51:50.053 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:50.051+0000 7fa3ce413040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.051+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 msgr2=0x7f631c0b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.051+0000 7f632094e640 1 --2- 172.21.15.113:0/464634316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b7420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f630c000c00 tx=0x7f630c0046c0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.051+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 shutdown_connections 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.051+0000 7f632094e640 1 --2- 172.21.15.113:0/464634316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f631c0b40e0 0x7f631c1550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.051+0000 7f632094e640 1 --2- 172.21.15.113:0/464634316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f631c0b3730 0x7f631c0ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.051+0000 7f632094e640 1 --2- 172.21.15.113:0/464634316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f631c0b3350 0x7f631c0b7420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.051+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 >> 172.21.15.113:0/464634316 conn(0x7f631c0d3030 msgr2=0x7f631c0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.052+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 shutdown_connections 2023-12-12T09:51:50.054 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.052+0000 7f632094e640 1 -- 172.21.15.113:0/464634316 wait complete. 2023-12-12T09:51:50.063 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:50.061+0000 7f7236412040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:51:50.071 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 633547c3-99bc-4bf2-b32b-d1a4dc2944df 2 2023-12-12T09:51:50.112 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:50.110+0000 7f7236412040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:51:50.173 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:50.170+0000 7fa3ce413040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:51:50.225 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:50.223+0000 7f7236412040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:51:50.256 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:50.254+0000 7fa3ce413040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:51:50.306 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:50.302+0000 7f7236412040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:51:50.307 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.305+0000 7feefbfff640 1 Processor -- start 2023-12-12T09:51:50.307 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.305+0000 7feefbfff640 1 -- start start 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.307+0000 7feefbfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b3350 0x7feefc0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.307+0000 7feefbfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7feefc0b3730 0x7feefc0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.307+0000 7feefbfff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0c6b80 con 0x7feefc0b40e0 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.307+0000 7feefbfff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc0d3bf0 con 0x7feefc0b3730 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.307+0000 7feefbfff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0d3ed0 con 0x7feefc0b3350 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefaffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b3350 0x7feefc0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefb7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feefc0b40e0 0x7feefc161200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42074/0 (socket says 172.21.15.113:42074) 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefaffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b3350 0x7feefc0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54536/0 (socket says 172.21.15.113:54536) 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefb7fe640 1 -- 172.21.15.113:0/1658929393 learned_addr learned my addr 172.21.15.113:0/1658929393 (peer_addr_for_me v1:172.21.15.113:0/0) 2023-12-12T09:51:50.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feef27fc640 1 --2- 172.21.15.113:0/1658929393 >> v2:172.21.15.72:3300/0 conn(0x7feefc0b3730 0x7feefc0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefaffd640 1 -- 172.21.15.113:0/1658929393 >> v2:172.21.15.72:3300/0 conn(0x7feefc0b3730 msgr2=0x7feefc0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefaffd640 1 --2- 172.21.15.113:0/1658929393 >> v2:172.21.15.72:3300/0 conn(0x7feefc0b3730 0x7feefc0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefaffd640 1 -- 172.21.15.113:0/1658929393 >> v1:172.21.15.113:6789/0 conn(0x7feefc0b40e0 legacy=0x7feefc161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefaffd640 1 -- 172.21.15.113:0/1658929393 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0d18c0 con 0x7feefc0b3350 2023-12-12T09:51:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.308+0000 7feefa7fc640 1 -- 172.21.15.113:0/1658929393 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682927953 0 0) 0x7feefc0c6b80 con 0x7feefc0b40e0 2023-12-12T09:51:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.309+0000 7feefaffd640 1 --2- 172.21.15.113:0/1658929393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b3350 0x7feefc0b3ba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feeec009d90 tx=0x7feeec02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe764c3c17734e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.309+0000 7feefa7fc640 1 -- 172.21.15.113:0/1658929393 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec02cc50 con 0x7feefc0b3350 2023-12-12T09:51:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.309+0000 7feefa7fc640 1 -- 172.21.15.113:0/1658929393 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec02cdb0 con 0x7feefc0b3350 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.309+0000 7feefa7fc640 1 -- 172.21.15.113:0/1658929393 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec034660 con 0x7feefc0b3350 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.310+0000 7feefbfff640 1 -- 172.21.15.113:0/1658929393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b3350 msgr2=0x7feefc0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.310+0000 7feefbfff640 1 --2- 172.21.15.113:0/1658929393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b3350 0x7feefc0b3ba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feeec009d90 tx=0x7feeec02bea0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.310+0000 7feefbfff640 1 -- 172.21.15.113:0/1658929393 shutdown_connections 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.310+0000 7feefbfff640 1 --2- 172.21.15.113:0/1658929393 >> v2:172.21.15.72:3300/0 conn(0x7feefc0b3730 0x7feefc0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.310+0000 7feefbfff640 1 --2- 172.21.15.113:0/1658929393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b3350 0x7feefc0b3ba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.310+0000 7feefbfff640 1 -- 172.21.15.113:0/1658929393 >> 172.21.15.113:0/1658929393 conn(0x7feefc0d3030 msgr2=0x7feefc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.310+0000 7feefbfff640 1 -- 172.21.15.113:0/1658929393 shutdown_connections 2023-12-12T09:51:50.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.310+0000 7feefbfff640 1 -- 172.21.15.113:0/1658929393 wait complete. 2023-12-12T09:51:50.314 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:50.312+0000 7fa3ce413040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:51:50.315 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.313+0000 7feefbfff640 1 Processor -- start 2023-12-12T09:51:50.315 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.313+0000 7feefbfff640 1 -- start start 2023-12-12T09:51:50.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feefbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feefc0b3350 0x7feefc159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feefbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feefc0b3730 0x7feefc0cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feefbfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b40e0 0x7feefc15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feefbfff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0caa90 con 0x7feefc0b3350 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feefbfff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc15a350 con 0x7feefc0b3730 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feefbfff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0ca1a0 con 0x7feefc0b40e0 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feef27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feefc0b3730 0x7feefc0cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feef27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feefc0b3730 0x7feefc0cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42032/0 (socket says 172.21.15.113:42032) 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feefb7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b40e0 0x7feefc15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feef27fc640 1 -- 172.21.15.113:0/348957656 learned_addr learned my addr 172.21.15.113:0/348957656 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feefaffd640 1 --2- 172.21.15.113:0/348957656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feefc0b3350 0x7feefc159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feef27fc640 1 -- 172.21.15.113:0/348957656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b40e0 msgr2=0x7feefc15efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feef27fc640 1 --2- 172.21.15.113:0/348957656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b40e0 0x7feefc15efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feef27fc640 1 -- 172.21.15.113:0/348957656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feefc0b3350 msgr2=0x7feefc159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feef27fc640 1 --2- 172.21.15.113:0/348957656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feefc0b3350 0x7feefc159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.315+0000 7feef27fc640 1 -- 172.21.15.113:0/348957656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec009a70 con 0x7feefc0b3730 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.316+0000 7feef27fc640 1 --2- 172.21.15.113:0/348957656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feefc0b3730 0x7feefc0cb260 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feee400d7d0 tx=0x7feee400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.316+0000 7feef3fff640 1 -- 172.21.15.113:0/348957656 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400b6c0 con 0x7feefc0b3730 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.316+0000 7feef3fff640 1 -- 172.21.15.113:0/348957656 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee400b820 con 0x7feefc0b3730 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.316+0000 7feef3fff640 1 -- 172.21.15.113:0/348957656 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400bbb0 con 0x7feefc0b3730 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.316+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc0b48a0 con 0x7feefc0b3730 2023-12-12T09:51:50.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.316+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc0ca720 con 0x7feefc0b3730 2023-12-12T09:51:50.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.317+0000 7feef3fff640 1 -- 172.21.15.113:0/348957656 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7feee401e760 con 0x7feefc0b3730 2023-12-12T09:51:50.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.317+0000 7feef3fff640 1 -- 172.21.15.113:0/348957656 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7feee401ee30 con 0x7feefc0b3730 2023-12-12T09:51:50.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.317+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc0cac00 con 0x7feefc0b3730 2023-12-12T09:51:50.324 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.322+0000 7feef3fff640 1 -- 172.21.15.113:0/348957656 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7feee401ea10 con 0x7feefc0b3730 2023-12-12T09:51:50.357 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:50.355+0000 7f7236412040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:51:50.378 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:50.376+0000 7fa3ce413040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:51:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.400+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "633547c3-99bc-4bf2-b32b-d1a4dc2944df", "id": 2} v 0) v1 -- 0x7feefc074450 con 0x7feefc0b3730 2023-12-12T09:51:50.409 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.407+0000 7feef3fff640 1 -- 172.21.15.113:0/348957656 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "633547c3-99bc-4bf2-b32b-d1a4dc2944df", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7feee4014520 con 0x7feefc0b3730 2023-12-12T09:51:50.409 INFO:teuthology.orchestra.run.smithi113.stdout:2 2023-12-12T09:51:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feefc0b3730 msgr2=0x7feefc0cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 --2- 172.21.15.113:0/348957656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feefc0b3730 0x7feefc0cb260 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feee400d7d0 tx=0x7feee400dc90 comp rx=0 tx=0).stop 2023-12-12T09:51:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 shutdown_connections 2023-12-12T09:51:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 --2- 172.21.15.113:0/348957656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feefc0b40e0 0x7feefc15efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 --2- 172.21.15.113:0/348957656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feefc0b3730 0x7feefc0cb260 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 --2- 172.21.15.113:0/348957656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feefc0b3350 0x7feefc159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 >> 172.21.15.113:0/348957656 conn(0x7feefc0d3030 msgr2=0x7feefc160650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 shutdown_connections 2023-12-12T09:51:50.412 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.408+0000 7feefbfff640 1 -- 172.21.15.113:0/348957656 wait complete. 2023-12-12T09:51:50.417 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:50.415+0000 7f7236412040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:51:50.428 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985 3 2023-12-12T09:51:50.437 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:50.435+0000 7fa3ce413040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:51:50.473 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:50.471+0000 7f7236412040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:51:50.620 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.618+0000 7ff68f0d1640 1 Processor -- start 2023-12-12T09:51:50.620 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.618+0000 7ff68f0d1640 1 -- start start 2023-12-12T09:51:50.622 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.619+0000 7ff68f0d1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6880cec20 0x7ff688167d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.619+0000 7ff68f0d1640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880b74d0 con 0x7ff6880b8970 2023-12-12T09:51:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68f0d1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880d1940 con 0x7ff68815d1f0 2023-12-12T09:51:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68f0d1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880b68e0 con 0x7ff6880cec20 2023-12-12T09:51:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68e0cf640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff68815d1f0 0x7ff6880d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42086/0 (socket says 172.21.15.113:42086) 2023-12-12T09:51:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68e0cf640 1 -- 172.21.15.113:0/3825289522 learned_addr learned my addr 172.21.15.113:0/3825289522 (peer_addr_for_me v1:172.21.15.113:0/0) 2023-12-12T09:51:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68e8d0640 1 --2- 172.21.15.113:0/3825289522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6880cec20 0x7ff688167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68d0cd640 1 -- 172.21.15.113:0/3825289522 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816819454 0 0) 0x7ff6880d1940 con 0x7ff68815d1f0 2023-12-12T09:51:50.623 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68d0cd640 1 -- 172.21.15.113:0/3825289522 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff674003520 con 0x7ff68815d1f0 2023-12-12T09:51:50.624 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68e8d0640 1 -- 172.21.15.113:0/3825289522 >> v1:172.21.15.113:6789/0 conn(0x7ff68815d1f0 legacy=0x7ff6880d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.624 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68e8d0640 1 -- 172.21.15.113:0/3825289522 >> v1:172.21.15.72:6789/0 conn(0x7ff6880b8970 legacy=0x7ff6880d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:50.624 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.620+0000 7ff68e8d0640 1 -- 172.21.15.113:0/3825289522 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880b6ba0 con 0x7ff6880cec20 2023-12-12T09:51:50.625 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.621+0000 7ff68e8d0640 1 --2- 172.21.15.113:0/3825289522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6880cec20 0x7ff688167d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff680009dc0 tx=0x7ff68002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23045e945ba9b9ff server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:50.625 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.622+0000 7ff68d0cd640 1 -- 172.21.15.113:0/3825289522 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68002cc40 con 0x7ff6880cec20 2023-12-12T09:51:50.625 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.622+0000 7ff68d0cd640 1 -- 172.21.15.113:0/3825289522 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68002cda0 con 0x7ff6880cec20 2023-12-12T09:51:50.625 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.622+0000 7ff68d0cd640 1 -- 172.21.15.113:0/3825289522 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680034660 con 0x7ff6880cec20 2023-12-12T09:51:50.625 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.622+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3825289522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6880cec20 msgr2=0x7ff688167d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.625 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.622+0000 7ff68f0d1640 1 --2- 172.21.15.113:0/3825289522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6880cec20 0x7ff688167d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff680009dc0 tx=0x7ff68002be90 comp rx=0 tx=0).stop 2023-12-12T09:51:50.626 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.622+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3825289522 shutdown_connections 2023-12-12T09:51:50.626 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.622+0000 7ff68f0d1640 1 --2- 172.21.15.113:0/3825289522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6880cec20 0x7ff688167d90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.626 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.622+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3825289522 >> 172.21.15.113:0/3825289522 conn(0x7ff6880d2fe0 msgr2=0x7ff6880d33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:50.626 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.623+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3825289522 shutdown_connections 2023-12-12T09:51:50.626 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.623+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3825289522 wait complete. 2023-12-12T09:51:50.627 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.624+0000 7ff68f0d1640 1 Processor -- start 2023-12-12T09:51:50.627 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.624+0000 7ff68f0d1640 1 -- start start 2023-12-12T09:51:50.629 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.625+0000 7ff68f0d1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6880b8970 0x7ff68815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.629 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.625+0000 7ff68f0d1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6880cec20 0x7ff68814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.629 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.625+0000 7ff68f0d1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff68815d1f0 0x7ff6880c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.629 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.625+0000 7ff68f0d1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815f330 con 0x7ff6880cec20 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.625+0000 7ff68f0d1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815dd50 con 0x7ff6880b8970 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.625+0000 7ff68f0d1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815e030 con 0x7ff68815d1f0 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff68815d1f0 0x7ff6880c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e0cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6880b8970 0x7ff68815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff68815d1f0 0x7ff6880c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54556/0 (socket says 172.21.15.113:54556) 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 -- 172.21.15.113:0/3206791551 learned_addr learned my addr 172.21.15.113:0/3206791551 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68d8ce640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6880cec20 0x7ff68814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 -- 172.21.15.113:0/3206791551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6880b8970 msgr2=0x7ff68815edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6880b8970 0x7ff68815edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 -- 172.21.15.113:0/3206791551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6880cec20 msgr2=0x7ff68814a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6880cec20 0x7ff68814a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.630 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 -- 172.21.15.113:0/3206791551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff680009a70 con 0x7ff68815d1f0 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68d8ce640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6880cec20 0x7ff68814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e0cf640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6880b8970 0x7ff68815edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.626+0000 7ff68e8d0640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff68815d1f0 0x7ff6880c8270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff680009ee0 tx=0x7ff680004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.627+0000 7ff67effd640 1 -- 172.21.15.113:0/3206791551 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680040650 con 0x7ff68815d1f0 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.627+0000 7ff67effd640 1 -- 172.21.15.113:0/3206791551 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6800407b0 con 0x7ff68815d1f0 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.627+0000 7ff67effd640 1 -- 172.21.15.113:0/3206791551 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680040b40 con 0x7ff68815d1f0 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.627+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68815e310 con 0x7ff68815d1f0 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.627+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68815e8d0 con 0x7ff68815d1f0 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.627+0000 7ff67effd640 1 -- 172.21.15.113:0/3206791551 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff680049400 con 0x7ff68815d1f0 2023-12-12T09:51:50.631 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.627+0000 7ff67effd640 1 -- 172.21.15.113:0/3206791551 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7ff68003e890 con 0x7ff68815d1f0 2023-12-12T09:51:50.632 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.628+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68815eaf0 con 0x7ff68815d1f0 2023-12-12T09:51:50.632 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.630+0000 7ff67effd640 1 -- 172.21.15.113:0/3206791551 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff680067ce0 con 0x7ff68815d1f0 2023-12-12T09:51:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.707+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985", "id": 3} v 0) v1 -- 0x7ff6880673f0 con 0x7ff68815d1f0 2023-12-12T09:51:50.717 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.715+0000 7ff67effd640 1 -- 172.21.15.113:0/3206791551 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff680056030 con 0x7ff68815d1f0 2023-12-12T09:51:50.717 INFO:teuthology.orchestra.run.smithi113.stdout:3 2023-12-12T09:51:50.719 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff68815d1f0 msgr2=0x7ff6880c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.720 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff68815d1f0 0x7ff6880c8270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff680009ee0 tx=0x7ff680004a40 comp rx=0 tx=0).stop 2023-12-12T09:51:50.720 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 shutdown_connections 2023-12-12T09:51:50.720 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff68815d1f0 0x7ff6880c8270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.720 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6880cec20 0x7ff68814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.720 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 --2- 172.21.15.113:0/3206791551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6880b8970 0x7ff68815edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.720 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 >> 172.21.15.113:0/3206791551 conn(0x7ff6880d2fe0 msgr2=0x7ff6880b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:50.720 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 shutdown_connections 2023-12-12T09:51:50.720 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.716+0000 7ff68f0d1640 1 -- 172.21.15.113:0/3206791551 wait complete. 2023-12-12T09:51:50.735 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 426889ad-4d63-416f-aeb4-71d485e31727 4 2023-12-12T09:51:50.911 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:50.909+0000 7fa3ce413040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:51:50.922 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:50.920+0000 7f7236412040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:51:50.970 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.967+0000 7f81cb789640 1 Processor -- start 2023-12-12T09:51:50.970 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.968+0000 7f81cb789640 1 -- start start 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.969+0000 7f81cb789640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f81c40d1510 0x7f81c40d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.969+0000 7f81cb789640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40b06b0 0x7f81c40b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.969+0000 7f81cb789640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c40cf130 con 0x7f81c415d620 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.969+0000 7f81cb789640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c40b0220 con 0x7f81c40d1510 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.969+0000 7f81cb789640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c415da00 con 0x7f81c40b06b0 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.969+0000 7f81c9d26640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40b06b0 0x7f81c40b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.970+0000 7f81c9d26640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40b06b0 0x7f81c40b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54558/0 (socket says 172.21.15.113:54558) 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.970+0000 7f81c9d26640 1 -- 172.21.15.113:0/2462819010 learned_addr learned my addr 172.21.15.113:0/2462819010 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:50.972 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.970+0000 7f81c8d24640 1 --2- 172.21.15.113:0/2462819010 >> v2:172.21.15.72:3300/0 conn(0x7f81c40d1510 0x7f81c40d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.973 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.970+0000 7f81c9d26640 1 -- 172.21.15.113:0/2462819010 >> v2:172.21.15.72:3300/0 conn(0x7f81c40d1510 msgr2=0x7f81c40d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:50.973 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.970+0000 7f81c9d26640 1 --2- 172.21.15.113:0/2462819010 >> v2:172.21.15.72:3300/0 conn(0x7f81c40d1510 0x7f81c40d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.973 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.970+0000 7f81c9d26640 1 -- 172.21.15.113:0/2462819010 >> v1:172.21.15.72:6789/0 conn(0x7f81c415d620 legacy=0x7f81c40d3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:50.973 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.970+0000 7f81c9d26640 1 -- 172.21.15.113:0/2462819010 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c40d1eb0 con 0x7f81c40b06b0 2023-12-12T09:51:50.973 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.971+0000 7f81c9d26640 1 --2- 172.21.15.113:0/2462819010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40b06b0 0x7f81c40b0b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009e50 tx=0x7f81bc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20435e47cce34ee server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:50.973 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.971+0000 7f81bbfff640 1 -- 172.21.15.113:0/2462819010 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc02ca70 con 0x7f81c40b06b0 2023-12-12T09:51:50.974 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.971+0000 7f81bbfff640 1 -- 172.21.15.113:0/2462819010 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81bc02cbd0 con 0x7f81c40b06b0 2023-12-12T09:51:50.974 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.971+0000 7f81bbfff640 1 -- 172.21.15.113:0/2462819010 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0343f0 con 0x7f81c40b06b0 2023-12-12T09:51:50.974 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.972+0000 7f81cb789640 1 -- 172.21.15.113:0/2462819010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40b06b0 msgr2=0x7f81c40b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.974 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.972+0000 7f81cb789640 1 --2- 172.21.15.113:0/2462819010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40b06b0 0x7f81c40b0b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009e50 tx=0x7f81bc02bed0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.975 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.972+0000 7f81cb789640 1 -- 172.21.15.113:0/2462819010 shutdown_connections 2023-12-12T09:51:50.975 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.972+0000 7f81cb789640 1 --2- 172.21.15.113:0/2462819010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40b06b0 0x7f81c40b0b40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.975 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.972+0000 7f81cb789640 1 --2- 172.21.15.113:0/2462819010 >> v2:172.21.15.72:3300/0 conn(0x7f81c40d1510 0x7f81c40d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.975 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.972+0000 7f81cb789640 1 -- 172.21.15.113:0/2462819010 >> 172.21.15.113:0/2462819010 conn(0x7f81c409c420 msgr2=0x7f81c40d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:50.975 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.972+0000 7f81cb789640 1 -- 172.21.15.113:0/2462819010 shutdown_connections 2023-12-12T09:51:50.975 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.972+0000 7f81cb789640 1 -- 172.21.15.113:0/2462819010 wait complete. 2023-12-12T09:51:50.976 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.974+0000 7f81cb789640 1 Processor -- start 2023-12-12T09:51:50.976 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.974+0000 7f81cb789640 1 -- start start 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81cb789640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81c40b06b0 0x7f81c4152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81cb789640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40d1510 0x7f81c40b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81cb789640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81c415d620 0x7f81c40b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81cb789640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40b7dd0 con 0x7f81c40b06b0 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81cb789640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c4160870 con 0x7f81c415d620 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81cb789640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40b8090 con 0x7f81c40d1510 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81c8d24640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40d1510 0x7f81c40b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81c9d26640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81c415d620 0x7f81c40b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81c8d24640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40d1510 0x7f81c40b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54564/0 (socket says 172.21.15.113:54564) 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81c8d24640 1 -- 172.21.15.113:0/2630716710 learned_addr learned my addr 172.21.15.113:0/2630716710 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.975+0000 7f81c9525640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81c40b06b0 0x7f81c4152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:50.979 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81c8d24640 1 -- 172.21.15.113:0/2630716710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81c415d620 msgr2=0x7f81c40b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81c8d24640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81c415d620 0x7f81c40b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81c8d24640 1 -- 172.21.15.113:0/2630716710 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81c40b06b0 msgr2=0x7f81c4152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81c8d24640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81c40b06b0 0x7f81c4152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81c8d24640 1 -- 172.21.15.113:0/2630716710 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc009b30 con 0x7f81c40d1510 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81c9525640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81c40b06b0 0x7f81c4152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81c8d24640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40d1510 0x7f81c40b61d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81b400d5f0 tx=0x7f81b400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81b9ffb640 1 -- 172.21.15.113:0/2630716710 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40125f0 con 0x7f81c40d1510 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81b9ffb640 1 -- 172.21.15.113:0/2630716710 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b4012750 con 0x7f81c40d1510 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c40b8370 con 0x7f81c40d1510 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81b9ffb640 1 -- 172.21.15.113:0/2630716710 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4012b00 con 0x7f81c40d1510 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.976+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c40b6fa0 con 0x7f81c40d1510 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.977+0000 7f81b9ffb640 1 -- 172.21.15.113:0/2630716710 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f81b401e460 con 0x7f81c40d1510 2023-12-12T09:51:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.977+0000 7f81b9ffb640 1 -- 172.21.15.113:0/2630716710 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f81b40149e0 con 0x7f81c40d1510 2023-12-12T09:51:50.981 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.977+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c4074450 con 0x7f81c40d1510 2023-12-12T09:51:50.982 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:50.980+0000 7f81b9ffb640 1 -- 172.21.15.113:0/2630716710 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f81b4014c30 con 0x7f81c40d1510 2023-12-12T09:51:51.060 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.057+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "426889ad-4d63-416f-aeb4-71d485e31727", "id": 4} v 0) v1 -- 0x7f81c40b5fb0 con 0x7f81c40d1510 2023-12-12T09:51:51.067 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.065+0000 7f81b9ffb640 1 -- 172.21.15.113:0/2630716710 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "426889ad-4d63-416f-aeb4-71d485e31727", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f81b40144d0 con 0x7f81c40d1510 2023-12-12T09:51:51.067 INFO:teuthology.orchestra.run.smithi113.stdout:4 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40d1510 msgr2=0x7f81c40b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40d1510 0x7f81c40b61d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81b400d5f0 tx=0x7f81b400dab0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 shutdown_connections 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81c415d620 0x7f81c40b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81c40d1510 0x7f81c40b61d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 --2- 172.21.15.113:0/2630716710 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81c40b06b0 0x7f81c4152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 >> 172.21.15.113:0/2630716710 conn(0x7f81c409c420 msgr2=0x7f81c40b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 shutdown_connections 2023-12-12T09:51:51.069 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.066+0000 7f81cb789640 1 -- 172.21.15.113:0/2630716710 wait complete. 2023-12-12T09:51:51.080 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:51.078+0000 7fa3ce413040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:51:51.081 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:51.080+0000 7f7236412040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:51:51.085 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new c3ee9923-a20a-4b0b-8ed1-b27acfd5582d 5 2023-12-12T09:51:51.206 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:51.204+0000 7f7236412040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:51:51.232 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:51.229+0000 7fa3ce413040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:51:51.259 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:51.257+0000 7f7236412040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:51:51.271 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.269+0000 7fddf3fff640 1 Processor -- start 2023-12-12T09:51:51.272 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.269+0000 7fddf3fff640 1 -- start start 2023-12-12T09:51:51.273 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.271+0000 7fddf3fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fddf40b3350 0x7fddf40b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.273 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.271+0000 7fddf3fff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40c6b80 con 0x7fddf40b3730 2023-12-12T09:51:51.273 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.271+0000 7fddf3fff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40d3bf0 con 0x7fddf40cfe70 2023-12-12T09:51:51.273 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.271+0000 7fddf3fff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf40d3ed0 con 0x7fddf40b3350 2023-12-12T09:51:51.273 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.271+0000 7fddf37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fddf40cfe70 0x7fddf40cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42102/0 (socket says 172.21.15.113:42102) 2023-12-12T09:51:51.274 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.271+0000 7fddf37fe640 1 -- 172.21.15.113:0/2672263355 learned_addr learned my addr 172.21.15.113:0/2672263355 (peer_addr_for_me v1:172.21.15.113:0/0) 2023-12-12T09:51:51.275 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/2672263355 >> v2:172.21.15.72:3300/0 conn(0x7fddf40b3350 0x7fddf40b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357005434 0 0) 0x7fddf40d3bf0 con 0x7fddf40cfe70 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdddc003520 con 0x7fddf40cfe70 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 264454807 0 0) 0x7fdddc003520 con 0x7fddf40cfe70 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 >> v2:172.21.15.72:3300/0 conn(0x7fddf40b3350 msgr2=0x7fddf40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 --2- 172.21.15.113:0/2672263355 >> v2:172.21.15.72:3300/0 conn(0x7fddf40b3350 0x7fddf40b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 >> v1:172.21.15.72:6789/0 conn(0x7fddf40b3730 legacy=0x7fddf40cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40d18c0 con 0x7fddf40cfe70 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fddec0019f0 con 0x7fddf40cfe70 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 >> v1:172.21.15.113:6789/0 conn(0x7fddf40cfe70 legacy=0x7fddf40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc003970 0x7fdddc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc0047c0 0x7fdddc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc005180 0x7fdddc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40d3ed0 con 0x7fdddc0047c0 2023-12-12T09:51:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc00c020 con 0x7fdddc003970 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.272+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc00c1e0 con 0x7fdddc005180 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc003970 0x7fdddc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf37fe640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc005180 0x7fdddc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf27fc640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc0047c0 0x7fdddc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf2ffd640 1 -- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc005180 msgr2=0x7fdddc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc005180 0x7fdddc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf2ffd640 1 -- 172.21.15.113:0/2672263355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc0047c0 msgr2=0x7fdddc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc0047c0 0x7fdddc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf2ffd640 1 -- 172.21.15.113:0/2672263355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc003520 con 0x7fdddc003970 2023-12-12T09:51:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.273+0000 7fddf37fe640 1 -- 172.21.15.113:0/2672263355 reap_dead start 2023-12-12T09:51:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.274+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc003970 0x7fdddc003d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009ac0 tx=0x7fdddc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bb804fe4dc01b90 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.274+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde402c6b0 con 0x7fdddc003970 2023-12-12T09:51:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.274+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde402c810 con 0x7fdddc003970 2023-12-12T09:51:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.274+0000 7fddf1ffb640 1 -- 172.21.15.113:0/2672263355 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde402cba0 con 0x7fdddc003970 2023-12-12T09:51:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.275+0000 7fddf3fff640 1 -- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc003970 msgr2=0x7fdddc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.275+0000 7fddf3fff640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc003970 0x7fdddc003d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009ac0 tx=0x7fdddc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.275+0000 7fddf3fff640 1 -- 172.21.15.113:0/2672263355 shutdown_connections 2023-12-12T09:51:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.275+0000 7fddf3fff640 1 --2- 172.21.15.113:0/2672263355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc003970 0x7fdddc003d90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.275+0000 7fddf3fff640 1 -- 172.21.15.113:0/2672263355 >> 172.21.15.113:0/2672263355 conn(0x7fddf40d3030 msgr2=0x7fddf40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.275+0000 7fddf3fff640 1 -- 172.21.15.113:0/2672263355 shutdown_connections 2023-12-12T09:51:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.275+0000 7fddf3fff640 1 -- 172.21.15.113:0/2672263355 wait complete. 2023-12-12T09:51:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.277+0000 7fddf3fff640 1 Processor -- start 2023-12-12T09:51:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.277+0000 7fddf3fff640 1 -- start start 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc003970 0x7fddf40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf41528e0 0x7fddf4158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf3fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddf4157940 0x7fddf4157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf3fff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4159230 con 0x7fdddc003970 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf3fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40b14a0 con 0x7fddf41528e0 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf3fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40b1780 con 0x7fddf4157940 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf41528e0 0x7fddf4158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf41528e0 0x7fddf4158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42054/0 (socket says 172.21.15.113:42054) 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf27fc640 1 -- 172.21.15.113:0/1354413173 learned_addr learned my addr 172.21.15.113:0/1354413173 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.278+0000 7fddf37fe640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddf4157940 0x7fddf4157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc003970 0x7fddf40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf27fc640 1 -- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddf4157940 msgr2=0x7fddf4157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf27fc640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddf4157940 0x7fddf4157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf27fc640 1 -- 172.21.15.113:0/1354413173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc003970 msgr2=0x7fddf40b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf27fc640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc003970 0x7fddf40b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf27fc640 1 -- 172.21.15.113:0/1354413173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40b48a0 con 0x7fddf41528e0 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc003970 0x7fddf40b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf37fe640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddf4157940 0x7fddf4157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fddf27fc640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf41528e0 0x7fddf4158cf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdde800e480 tx=0x7fdde800e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.279+0000 7fdddb7fe640 1 -- 172.21.15.113:0/1354413173 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde80052e0 con 0x7fddf41528e0 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.280+0000 7fdddb7fe640 1 -- 172.21.15.113:0/1354413173 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde8005440 con 0x7fddf41528e0 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.280+0000 7fdddb7fe640 1 -- 172.21.15.113:0/1354413173 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8003830 con 0x7fddf41528e0 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.280+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf40b1a60 con 0x7fddf41528e0 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.280+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf415d340 con 0x7fddf41528e0 2023-12-12T09:51:51.283 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.280+0000 7fdddb7fe640 1 -- 172.21.15.113:0/1354413173 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdde8003c90 con 0x7fddf41528e0 2023-12-12T09:51:51.284 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.280+0000 7fdddb7fe640 1 -- 172.21.15.113:0/1354413173 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fdde8013480 con 0x7fddf41528e0 2023-12-12T09:51:51.284 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.281+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf415d500 con 0x7fddf41528e0 2023-12-12T09:51:51.285 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.283+0000 7fdddb7fe640 1 -- 172.21.15.113:0/1354413173 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fdde801d460 con 0x7fddf41528e0 2023-12-12T09:51:51.289 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:51.287+0000 7fa3ce413040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:51:51.363 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.361+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c3ee9923-a20a-4b0b-8ed1-b27acfd5582d", "id": 5} v 0) v1 -- 0x7fddf40666e0 con 0x7fddf41528e0 2023-12-12T09:51:51.368 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:51.367+0000 7f7236412040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:51:51.369 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.367+0000 7fdddb7fe640 1 -- 172.21.15.113:0/1354413173 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3ee9923-a20a-4b0b-8ed1-b27acfd5582d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdde802c0b0 con 0x7fddf41528e0 2023-12-12T09:51:51.370 INFO:teuthology.orchestra.run.smithi113.stdout:5 2023-12-12T09:51:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf41528e0 msgr2=0x7fddf4158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf41528e0 0x7fddf4158cf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdde800e480 tx=0x7fdde800e940 comp rx=0 tx=0).stop 2023-12-12T09:51:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 shutdown_connections 2023-12-12T09:51:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddf4157940 0x7fddf4157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf41528e0 0x7fddf4158cf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 --2- 172.21.15.113:0/1354413173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdddc003970 0x7fddf40b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.372 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 >> 172.21.15.113:0/1354413173 conn(0x7fddf40d3030 msgr2=0x7fddf40c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:51.372 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 shutdown_connections 2023-12-12T09:51:51.372 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.369+0000 7fddf3fff640 1 -- 172.21.15.113:0/1354413173 wait complete. 2023-12-12T09:51:51.388 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 5a95b4bf-c94b-4246-ab6d-ac5650b885f0 6 2023-12-12T09:51:51.405 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:51.403+0000 7fa3ce413040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:51:51.482 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:51.480+0000 7f7236412040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:51:51.522 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:51.519+0000 7fa3ce413040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:51:51.571 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.568+0000 7f9ba194e640 1 Processor -- start 2023-12-12T09:51:51.571 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.568+0000 7f9ba194e640 1 -- start start 2023-12-12T09:51:51.572 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9ba194e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9b9c155400 0x7f9b9c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.572 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9ba194e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.572 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9ba194e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b9c0cedf0 0x7f9b9c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.572 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9ba194e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c0cf7c0 con 0x7f9b9c155400 2023-12-12T09:51:51.572 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9ba194e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c15b4a0 con 0x7f9b9c0cedf0 2023-12-12T09:51:51.573 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9ba194e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0adbb0 con 0x7f9b9c0ce150 2023-12-12T09:51:51.573 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.573 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c0ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54582/0 (socket says 172.21.15.113:54582) 2023-12-12T09:51:51.573 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9b9bfff640 1 -- 172.21.15.113:0/3634737124 learned_addr learned my addr 172.21.15.113:0/3634737124 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:51.573 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9ba114d640 1 --2- 172.21.15.113:0/3634737124 >> v2:172.21.15.113:3300/0 conn(0x7f9b9c0cedf0 0x7f9b9c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.573 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9ba094c640 1 --2- 172.21.15.113:0/3634737124 >> v2:172.21.15.72:3300/0 conn(0x7f9b9c155400 0x7f9b9c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.570+0000 7f9b9bfff640 1 -- 172.21.15.113:0/3634737124 >> v2:172.21.15.113:3300/0 conn(0x7f9b9c0cedf0 msgr2=0x7f9b9c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.571+0000 7f9b9bfff640 1 --2- 172.21.15.113:0/3634737124 >> v2:172.21.15.113:3300/0 conn(0x7f9b9c0cedf0 0x7f9b9c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.571+0000 7f9b9bfff640 1 -- 172.21.15.113:0/3634737124 >> v2:172.21.15.72:3300/0 conn(0x7f9b9c155400 msgr2=0x7f9b9c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.571+0000 7f9b9bfff640 1 --2- 172.21.15.113:0/3634737124 >> v2:172.21.15.72:3300/0 conn(0x7f9b9c155400 0x7f9b9c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.571+0000 7f9b9bfff640 1 -- 172.21.15.113:0/3634737124 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0ade90 con 0x7f9b9c0ce150 2023-12-12T09:51:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.572+0000 7f9b9bfff640 1 --2- 172.21.15.113:0/3634737124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c0ce5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c06cf30 tx=0x7f9b8402bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbc64f580224366 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:51.575 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.572+0000 7f9b9b7fe640 1 -- 172.21.15.113:0/3634737124 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8402ca80 con 0x7f9b9c0ce150 2023-12-12T09:51:51.575 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.572+0000 7f9b9b7fe640 1 -- 172.21.15.113:0/3634737124 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8402cbe0 con 0x7f9b9c0ce150 2023-12-12T09:51:51.575 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.572+0000 7f9b9b7fe640 1 -- 172.21.15.113:0/3634737124 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840343f0 con 0x7f9b9c0ce150 2023-12-12T09:51:51.575 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 -- 172.21.15.113:0/3634737124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 msgr2=0x7f9b9c0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.575 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 --2- 172.21.15.113:0/3634737124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c0ce5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c06cf30 tx=0x7f9b8402bee0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.575 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 -- 172.21.15.113:0/3634737124 shutdown_connections 2023-12-12T09:51:51.576 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 --2- 172.21.15.113:0/3634737124 >> v2:172.21.15.113:3300/0 conn(0x7f9b9c0cedf0 0x7f9b9c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.576 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 --2- 172.21.15.113:0/3634737124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c0ce5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.576 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 --2- 172.21.15.113:0/3634737124 >> v2:172.21.15.72:3300/0 conn(0x7f9b9c155400 0x7f9b9c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.576 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 -- 172.21.15.113:0/3634737124 >> 172.21.15.113:0/3634737124 conn(0x7f9b9c06a640 msgr2=0x7f9b9c06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:51.576 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 -- 172.21.15.113:0/3634737124 shutdown_connections 2023-12-12T09:51:51.576 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.573+0000 7f9ba194e640 1 -- 172.21.15.113:0/3634737124 wait complete. 2023-12-12T09:51:51.577 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.575+0000 7f9ba194e640 1 Processor -- start 2023-12-12T09:51:51.577 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.575+0000 7f9ba194e640 1 -- start start 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba194e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c1465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba194e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b9c0cedf0 0x7f9b9c1e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba194e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b9c155400 0x7f9b9c159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba194e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1f2d70 con 0x7f9b9c0cedf0 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba194e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1e4890 con 0x7f9b9c155400 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba194e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1e4b70 con 0x7f9b9c0ce150 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba094c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c1465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba094c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c1465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54586/0 (socket says 172.21.15.113:54586) 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba094c640 1 -- 172.21.15.113:0/1303532396 learned_addr learned my addr 172.21.15.113:0/1303532396 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9ba114d640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b9c155400 0x7f9b9c159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.580 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.576+0000 7f9b9bfff640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b9c0cedf0 0x7f9b9c1e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.580 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9ba094c640 1 -- 172.21.15.113:0/1303532396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b9c155400 msgr2=0x7f9b9c159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.580 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9ba094c640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b9c155400 0x7f9b9c159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.580 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9ba094c640 1 -- 172.21.15.113:0/1303532396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b9c0cedf0 msgr2=0x7f9b9c1e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.580 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9ba094c640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b9c0cedf0 0x7f9b9c1e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.580 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9ba094c640 1 -- 172.21.15.113:0/1303532396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b84009b30 con 0x7f9b9c0ce150 2023-12-12T09:51:51.580 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9ba094c640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c1465d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00d5f0 tx=0x7f9b8c00dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9b997fa640 1 -- 172.21.15.113:0/1303532396 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0125f0 con 0x7f9b9c0ce150 2023-12-12T09:51:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9b997fa640 1 -- 172.21.15.113:0/1303532396 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c012750 con 0x7f9b9c0ce150 2023-12-12T09:51:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9b997fa640 1 -- 172.21.15.113:0/1303532396 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c012b00 con 0x7f9b9c0ce150 2023-12-12T09:51:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c1e4e50 con 0x7f9b9c0ce150 2023-12-12T09:51:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.577+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c1e5130 con 0x7f9b9c0ce150 2023-12-12T09:51:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.578+0000 7f9b997fa640 1 -- 172.21.15.113:0/1303532396 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9b8c01e460 con 0x7f9b9c0ce150 2023-12-12T09:51:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.578+0000 7f9b997fa640 1 -- 172.21.15.113:0/1303532396 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f9b8c01ed00 con 0x7f9b9c0ce150 2023-12-12T09:51:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.579+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c1f2b90 con 0x7f9b9c0ce150 2023-12-12T09:51:51.583 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.581+0000 7f9b997fa640 1 -- 172.21.15.113:0/1303532396 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9b8c01e710 con 0x7f9b9c0ce150 2023-12-12T09:51:51.642 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:51.640+0000 7f7236412040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:51:51.661 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.658+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a95b4bf-c94b-4246-ab6d-ac5650b885f0", "id": 6} v 0) v1 -- 0x7f9b9c074450 con 0x7f9b9c0ce150 2023-12-12T09:51:51.668 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.666+0000 7f9b997fa640 1 -- 172.21.15.113:0/1303532396 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a95b4bf-c94b-4246-ab6d-ac5650b885f0", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9b8c01e9e0 con 0x7f9b9c0ce150 2023-12-12T09:51:51.668 INFO:teuthology.orchestra.run.smithi113.stdout:6 2023-12-12T09:51:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 msgr2=0x7f9b9c1465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c1465d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00d5f0 tx=0x7f9b8c00dab0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 shutdown_connections 2023-12-12T09:51:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b9c155400 0x7f9b9c159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b9c0cedf0 0x7f9b9c1e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 --2- 172.21.15.113:0/1303532396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b9c0ce150 0x7f9b9c1465d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 >> 172.21.15.113:0/1303532396 conn(0x7f9b9c06a640 msgr2=0x7f9b9c15b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 shutdown_connections 2023-12-12T09:51:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.667+0000 7f9ba194e640 1 -- 172.21.15.113:0/1303532396 wait complete. 2023-12-12T09:51:51.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 1ada7456-408c-413b-a457-a982a576b8a1 7 2023-12-12T09:51:51.688 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:51.686+0000 7fa3ce413040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:51:51.691 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:51.690+0000 7f7236412040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:51:51.741 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:51.739+0000 7fa3ce413040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:51:51.850 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:51.848+0000 7f7236412040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:51:51.874 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.872+0000 7f8d9e83d640 1 Processor -- start 2023-12-12T09:51:51.874 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.872+0000 7f8d9e83d640 1 -- start start 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.873+0000 7f8d9e83d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d9815d620 0x7f8d980d3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d9e83d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d9e83d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d980cf130 con 0x7f8d980cf810 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d9e83d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d980b0220 con 0x7f8d9815d620 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d9e83d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9815da00 con 0x7f8d980d1510 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d977fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d97fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d9815d620 0x7f8d980d3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d977fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:54602/0 (socket says 172.21.15.113:54602) 2023-12-12T09:51:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d977fe640 1 -- 172.21.15.113:0/2179323690 learned_addr learned my addr 172.21.15.113:0/2179323690 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:51.877 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d977fe640 1 -- 172.21.15.113:0/2179323690 >> v2:172.21.15.113:3300/0 conn(0x7f8d9815d620 msgr2=0x7f8d980d3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:51.877 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d977fe640 1 --2- 172.21.15.113:0/2179323690 >> v2:172.21.15.113:3300/0 conn(0x7f8d9815d620 0x7f8d980d3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.877 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d977fe640 1 -- 172.21.15.113:0/2179323690 >> v1:172.21.15.72:6789/0 conn(0x7f8d980cf810 legacy=0x7f8d980cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:51.877 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.874+0000 7f8d977fe640 1 -- 172.21.15.113:0/2179323690 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d980d1eb0 con 0x7f8d980d1510 2023-12-12T09:51:51.878 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.875+0000 7f8d977fe640 1 --2- 172.21.15.113:0/2179323690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980d1970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009870 tx=0x7f8d8c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7884d4f6cf650dc server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:51.878 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d96ffd640 1 -- 172.21.15.113:0/2179323690 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c031640 con 0x7f8d980d1510 2023-12-12T09:51:51.879 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d96ffd640 1 -- 172.21.15.113:0/2179323690 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c0317a0 con 0x7f8d980d1510 2023-12-12T09:51:51.879 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d96ffd640 1 -- 172.21.15.113:0/2179323690 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c031b30 con 0x7f8d980d1510 2023-12-12T09:51:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d9e83d640 1 -- 172.21.15.113:0/2179323690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 msgr2=0x7f8d980d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d9e83d640 1 --2- 172.21.15.113:0/2179323690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980d1970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009870 tx=0x7f8d8c02b8f0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d9e83d640 1 -- 172.21.15.113:0/2179323690 shutdown_connections 2023-12-12T09:51:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d9e83d640 1 --2- 172.21.15.113:0/2179323690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980d1970 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d9e83d640 1 --2- 172.21.15.113:0/2179323690 >> v2:172.21.15.113:3300/0 conn(0x7f8d9815d620 0x7f8d980d3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.876+0000 7f8d9e83d640 1 -- 172.21.15.113:0/2179323690 >> 172.21.15.113:0/2179323690 conn(0x7f8d9809c420 msgr2=0x7f8d980d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.877+0000 7f8d9e83d640 1 -- 172.21.15.113:0/2179323690 shutdown_connections 2023-12-12T09:51:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.877+0000 7f8d9e83d640 1 -- 172.21.15.113:0/2179323690 wait complete. 2023-12-12T09:51:51.881 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.878+0000 7f8d9e83d640 1 Processor -- start 2023-12-12T09:51:51.881 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.878+0000 7f8d9e83d640 1 -- start start 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.879+0000 7f8d9e83d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d980cf810 0x7f8d980b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.879+0000 7f8d9e83d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.879+0000 7f8d9e83d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d9815d620 0x7f8d980b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.879+0000 7f8d9e83d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d98157f20 con 0x7f8d9815d620 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.879+0000 7f8d9e83d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d980c9b50 con 0x7f8d980cf810 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.879+0000 7f8d9e83d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9815ed30 con 0x7f8d980d1510 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.879+0000 7f8d97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d980cf810 0x7f8d980b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d980cf810 0x7f8d980b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42084/0 (socket says 172.21.15.113:42084) 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d97fff640 1 -- 172.21.15.113:0/40424266 learned_addr learned my addr 172.21.15.113:0/40424266 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-12-12T09:51:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d977fe640 1 --2- 172.21.15.113:0/40424266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:51.883 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d97fff640 1 -- 172.21.15.113:0/40424266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 msgr2=0x7f8d980c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.883 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d97fff640 1 --2- 172.21.15.113:0/40424266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.883 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d97fff640 1 -- 172.21.15.113:0/40424266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d9815d620 msgr2=0x7f8d980b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.883 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d97fff640 1 --2- 172.21.15.113:0/40424266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d9815d620 0x7f8d980b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.883 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d97fff640 1 -- 172.21.15.113:0/40424266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0094d0 con 0x7f8d980cf810 2023-12-12T09:51:51.884 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d97fff640 1 --2- 172.21.15.113:0/40424266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d980cf810 0x7f8d980b5d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800e9b0 tx=0x7f8d8800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:51.884 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d94ff9640 1 -- 172.21.15.113:0/40424266 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88009670 con 0x7f8d980cf810 2023-12-12T09:51:51.884 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d94ff9640 1 -- 172.21.15.113:0/40424266 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8800c9a0 con 0x7f8d980cf810 2023-12-12T09:51:51.884 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d94ff9640 1 -- 172.21.15.113:0/40424266 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8800cd50 con 0x7f8d980cf810 2023-12-12T09:51:51.884 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.880+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9815f010 con 0x7f8d980cf810 2023-12-12T09:51:51.884 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.881+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9815f5d0 con 0x7f8d980cf810 2023-12-12T09:51:51.885 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.881+0000 7f8d94ff9640 1 -- 172.21.15.113:0/40424266 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8d88005c90 con 0x7f8d980cf810 2023-12-12T09:51:51.885 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.881+0000 7f8d94ff9640 1 -- 172.21.15.113:0/40424266 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f8d88013470 con 0x7f8d980cf810 2023-12-12T09:51:51.885 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.882+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d98074450 con 0x7f8d980cf810 2023-12-12T09:51:51.886 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.884+0000 7f8d94ff9640 1 -- 172.21.15.113:0/40424266 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8d88021060 con 0x7f8d980cf810 2023-12-12T09:51:51.908 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:51.906+0000 7fa3ce413040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:51:51.964 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.962+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1ada7456-408c-413b-a457-a982a576b8a1", "id": 7} v 0) v1 -- 0x7f8d980c6770 con 0x7f8d980cf810 2023-12-12T09:51:51.975 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.973+0000 7f8d94ff9640 1 -- 172.21.15.113:0/40424266 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ada7456-408c-413b-a457-a982a576b8a1", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8d88029020 con 0x7f8d980cf810 2023-12-12T09:51:51.975 INFO:teuthology.orchestra.run.smithi113.stdout:7 2023-12-12T09:51:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d980cf810 msgr2=0x7f8d980b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 --2- 172.21.15.113:0/40424266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d980cf810 0x7f8d980b5d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800e9b0 tx=0x7f8d8800ee70 comp rx=0 tx=0).stop 2023-12-12T09:51:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 shutdown_connections 2023-12-12T09:51:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 --2- 172.21.15.113:0/40424266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d9815d620 0x7f8d980b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 --2- 172.21.15.113:0/40424266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d980d1510 0x7f8d980c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 --2- 172.21.15.113:0/40424266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d980cf810 0x7f8d980b5d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 >> 172.21.15.113:0/40424266 conn(0x7f8d9809c420 msgr2=0x7f8d980cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:51.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 shutdown_connections 2023-12-12T09:51:51.978 INFO:teuthology.orchestra.run.smithi113.stderr:2023-12-12T09:51:51.974+0000 7f8d9e83d640 1 -- 172.21.15.113:0/40424266 wait complete. 2023-12-12T09:51:51.993 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-12T09:51:51.993 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-12T09:51:52.002 INFO:tasks.ceph.osd.0:Started 2023-12-12T09:51:52.002 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-12T09:51:52.003 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-12T09:51:52.011 INFO:tasks.ceph.osd.1:Started 2023-12-12T09:51:52.012 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-12T09:51:52.012 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-12T09:51:52.017 INFO:tasks.ceph.osd.2:Started 2023-12-12T09:51:52.017 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-12T09:51:52.017 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-12T09:51:52.022 INFO:tasks.ceph.osd.3:Started 2023-12-12T09:51:52.023 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-12T09:51:52.023 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-12T09:51:52.027 INFO:tasks.ceph.osd.4:Started 2023-12-12T09:51:52.028 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-12T09:51:52.028 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-12T09:51:52.036 INFO:tasks.ceph.osd.5:Started 2023-12-12T09:51:52.036 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-12T09:51:52.036 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-12T09:51:52.041 INFO:tasks.ceph.osd.6:Started 2023-12-12T09:51:52.041 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-12T09:51:52.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-12T09:51:52.047 INFO:tasks.ceph.osd.7:Started 2023-12-12T09:51:52.047 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:51:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.271+0000 7f0bb2cad640 1 Processor -- start 2023-12-12T09:51:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.271+0000 7f0bb2cad640 1 -- start start 2023-12-12T09:51:52.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb2cad640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac157a60 0x7f0bac06c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:52.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb2cad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0bac06de20 0x7f0bac15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:52.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb2cad640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0bac15e0e0 0x7f0bac15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:52.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb2cad640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac15e550 con 0x7f0bac15e0e0 2023-12-12T09:51:52.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb2cad640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac06ca40 con 0x7f0bac06de20 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb2cad640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac071cd0 con 0x7f0bac157a60 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb124a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0bac15e0e0 0x7f0bac15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb0a49640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac157a60 0x7f0bac06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb124a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0bac15e0e0 0x7f0bac15a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41908/0 (socket says 172.21.15.72:41908) 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb124a640 1 -- 172.21.15.72:0/37080968 learned_addr learned my addr 172.21.15.72:0/37080968 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0babfff640 1 --2- 172.21.15.72:0/37080968 >> v2:172.21.15.113:3300/0 conn(0x7f0bac06de20 0x7f0bac15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb124a640 1 -- 172.21.15.72:0/37080968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac157a60 msgr2=0x7f0bac06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb124a640 1 --2- 172.21.15.72:0/37080968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac157a60 0x7f0bac06c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb124a640 1 -- 172.21.15.72:0/37080968 >> v2:172.21.15.113:3300/0 conn(0x7f0bac06de20 msgr2=0x7f0bac15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb124a640 1 --2- 172.21.15.72:0/37080968 >> v2:172.21.15.113:3300/0 conn(0x7f0bac06de20 0x7f0bac15dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.274+0000 7f0bb124a640 1 -- 172.21.15.72:0/37080968 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac070f10 con 0x7f0bac15e0e0 2023-12-12T09:51:52.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.275+0000 7f0bb124a640 1 --2- 172.21.15.72:0/37080968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac15a5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06b810 tx=0x7f0ba402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d826e91465dd1b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:52.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.276+0000 7f0bab7fe640 1 -- 172.21.15.72:0/37080968 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba402c9f0 con 0x7f0bac15e0e0 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.276+0000 7f0bab7fe640 1 -- 172.21.15.72:0/37080968 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba402cb50 con 0x7f0bac15e0e0 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.276+0000 7f0bab7fe640 1 -- 172.21.15.72:0/37080968 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40343f0 con 0x7f0bac15e0e0 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.276+0000 7f0bb2cad640 1 -- 172.21.15.72:0/37080968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 msgr2=0x7f0bac15a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.276+0000 7f0bb2cad640 1 --2- 172.21.15.72:0/37080968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac15a5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06b810 tx=0x7f0ba402be50 comp rx=0 tx=0).stop 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.277+0000 7f0bb2cad640 1 -- 172.21.15.72:0/37080968 shutdown_connections 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.277+0000 7f0bb2cad640 1 --2- 172.21.15.72:0/37080968 >> v2:172.21.15.113:3300/0 conn(0x7f0bac06de20 0x7f0bac15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.277+0000 7f0bb2cad640 1 --2- 172.21.15.72:0/37080968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac15a5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.277+0000 7f0bb2cad640 1 --2- 172.21.15.72:0/37080968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac157a60 0x7f0bac06c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.277+0000 7f0bb2cad640 1 -- 172.21.15.72:0/37080968 >> 172.21.15.72:0/37080968 conn(0x7f0bac0b0a90 msgr2=0x7f0bac0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.277+0000 7f0bb2cad640 1 -- 172.21.15.72:0/37080968 shutdown_connections 2023-12-12T09:51:52.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.277+0000 7f0bb2cad640 1 -- 172.21.15.72:0/37080968 wait complete. 2023-12-12T09:51:52.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.278+0000 7f0bb2cad640 1 Processor -- start 2023-12-12T09:51:52.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.278+0000 7f0bb2cad640 1 -- start start 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.279+0000 7f0bb2cad640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac06de20 0x7f0bac159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.279+0000 7f0bb2cad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bac157a60 0x7f0bac151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.279+0000 7f0bb2cad640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.279+0000 7f0bb2cad640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac159da0 con 0x7f0bac15e0e0 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.279+0000 7f0bb2cad640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac06d3e0 con 0x7f0bac157a60 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.279+0000 7f0bb2cad640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac06d6c0 con 0x7f0bac06de20 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41924/0 (socket says 172.21.15.72:41924) 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 -- 172.21.15.72:0/562067495 learned_addr learned my addr 172.21.15.72:0/562067495 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb0a49640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac06de20 0x7f0bac159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0babfff640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bac157a60 0x7f0bac151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 -- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac06de20 msgr2=0x7f0bac159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac06de20 0x7f0bac159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 -- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bac157a60 msgr2=0x7f0bac151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bac157a60 0x7f0bac151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 -- 172.21.15.72:0/562067495 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4009b30 con 0x7f0bac15e0e0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb0a49640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac06de20 0x7f0bac159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.280+0000 7f0bb124a640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac157e40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4036040 tx=0x7f0ba4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.281+0000 7f0ba97fa640 1 -- 172.21.15.72:0/562067495 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4009510 con 0x7f0bac15e0e0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.281+0000 7f0ba97fa640 1 -- 172.21.15.72:0/562067495 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba403e450 con 0x7f0bac15e0e0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.281+0000 7f0ba97fa640 1 -- 172.21.15.72:0/562067495 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba403e800 con 0x7f0bac15e0e0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.281+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac0c3d00 con 0x7f0bac15e0e0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.281+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac0c42c0 con 0x7f0bac15e0e0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.281+0000 7f0ba97fa640 1 -- 172.21.15.72:0/562067495 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0ba403ec60 con 0x7f0bac15e0e0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.281+0000 7f0ba97fa640 1 -- 172.21.15.72:0/562067495 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0ba4048400 con 0x7f0bac15e0e0 2023-12-12T09:51:52.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.282+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac0745d0 con 0x7f0bac15e0e0 2023-12-12T09:51:52.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.285+0000 7f0ba97fa640 1 -- 172.21.15.72:0/562067495 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f0ba4043020 con 0x7f0bac15e0e0 2023-12-12T09:51:52.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.348+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bac070860 con 0x7f0bac15e0e0 2023-12-12T09:51:52.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.349+0000 7f0ba97fa640 1 -- 172.21.15.72:0/562067495 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0ba4051430 con 0x7f0bac15e0e0 2023-12-12T09:51:52.351 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:51:52.351 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":10,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:51:51.965427+0000","last_up_change":"0.000000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:51:52.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.350+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 msgr2=0x7f0bac157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:52.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.350+0000 7f0bb2cad640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac157e40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4036040 tx=0x7f0ba4004a40 comp rx=0 tx=0).stop 2023-12-12T09:51:52.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.350+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 shutdown_connections 2023-12-12T09:51:52.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.351+0000 7f0bb2cad640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bac15e0e0 0x7f0bac157e40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.351+0000 7f0bb2cad640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bac157a60 0x7f0bac151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.351+0000 7f0bb2cad640 1 --2- 172.21.15.72:0/562067495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bac06de20 0x7f0bac159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:52.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.351+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 >> 172.21.15.72:0/562067495 conn(0x7f0bac0b0a90 msgr2=0x7f0bac15adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:52.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.351+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 shutdown_connections 2023-12-12T09:51:52.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:51:52.351+0000 7f0bb2cad640 1 -- 172.21.15.72:0/562067495 wait complete. 2023-12-12T09:51:52.370 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-12T09:51:52.370 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-12T09:51:52.716 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-12-12T09:51:52.714+0000 7fb4abec2500 -1 Falling back to public interface 2023-12-12T09:51:52.721 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-12-12T09:51:52.719+0000 7f5ff8ca7500 -1 Falling back to public interface 2023-12-12T09:51:52.727 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-12-12T09:51:52.726+0000 7fcde50b2500 -1 Falling back to public interface 2023-12-12T09:51:52.728 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-12-12T09:51:52.726+0000 7f0ff8c13500 -1 Falling back to public interface 2023-12-12T09:51:52.734 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-12-12T09:51:52.732+0000 7f2498438500 -1 Falling back to public interface 2023-12-12T09:51:52.737 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-12-12T09:51:52.735+0000 7fa3a294a500 -1 Falling back to public interface 2023-12-12T09:51:52.741 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-12-12T09:51:52.739+0000 7febabca7500 -1 Falling back to public interface 2023-12-12T09:51:52.766 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-12-12T09:51:52.762+0000 7f9feeeaf500 -1 Falling back to public interface 2023-12-12T09:51:53.024 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:53.022+0000 7f7236412040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:51:53.038 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:53.036+0000 7fa3ce413040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:51:53.178 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-12T09:51:53.330 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.328+0000 7efce6c09640 1 Processor -- start 2023-12-12T09:51:53.330 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.328+0000 7efce6c09640 1 -- start start 2023-12-12T09:51:53.331 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce6c09640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efce015d380 0x7efce00afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:53.332 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce6c09640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:53.332 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce6c09640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00b2c40 con 0x7efce00d1550 2023-12-12T09:51:53.332 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce6c09640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00b38c0 con 0x7efce015d380 2023-12-12T09:51:53.332 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce6c09640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00b2f20 con 0x7efce00ce630 2023-12-12T09:51:53.332 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce6408640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7efce00d1550 0x7efce00b3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43618/0 (socket says 172.21.15.72:43618) 2023-12-12T09:51:53.332 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce6408640 1 -- 172.21.15.72:0/3344029006 learned_addr learned my addr 172.21.15.72:0/3344029006 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:51:53.332 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce5c07640 1 --2- 172.21.15.72:0/3344029006 >> v2:172.21.15.113:3300/0 conn(0x7efce015d380 0x7efce00afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:53.332 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce5406640 1 --2- 172.21.15.72:0/3344029006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce4c05640 1 -- 172.21.15.72:0/3344029006 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122827919 0 0) 0x7efce00b2c40 con 0x7efce00d1550 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.330+0000 7efce4c05640 1 -- 172.21.15.72:0/3344029006 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efccc003520 con 0x7efce00d1550 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.331+0000 7efce5406640 1 -- 172.21.15.72:0/3344029006 >> v2:172.21.15.113:3300/0 conn(0x7efce015d380 msgr2=0x7efce00afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.331+0000 7efce5406640 1 --2- 172.21.15.72:0/3344029006 >> v2:172.21.15.113:3300/0 conn(0x7efce015d380 0x7efce00afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.331+0000 7efce5406640 1 -- 172.21.15.72:0/3344029006 >> v1:172.21.15.72:6789/0 conn(0x7efce00d1550 legacy=0x7efce00b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.331+0000 7efce5406640 1 -- 172.21.15.72:0/3344029006 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00b3200 con 0x7efce00ce630 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.332+0000 7efce5406640 1 --2- 172.21.15.72:0/3344029006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce00cea70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efcd00097a0 tx=0x7efcd002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03094b82a28dd49 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.332+0000 7efce4c05640 1 -- 172.21.15.72:0/3344029006 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0031780 con 0x7efce00ce630 2023-12-12T09:51:53.334 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.332+0000 7efce4c05640 1 -- 172.21.15.72:0/3344029006 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd00318e0 con 0x7efce00ce630 2023-12-12T09:51:53.335 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.332+0000 7efce4c05640 1 -- 172.21.15.72:0/3344029006 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0031c70 con 0x7efce00ce630 2023-12-12T09:51:53.335 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.332+0000 7efce6c09640 1 -- 172.21.15.72:0/3344029006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 msgr2=0x7efce00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:53.335 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.332+0000 7efce6c09640 1 --2- 172.21.15.72:0/3344029006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce00cea70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efcd00097a0 tx=0x7efcd002b820 comp rx=0 tx=0).stop 2023-12-12T09:51:53.336 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.333+0000 7efce6c09640 1 -- 172.21.15.72:0/3344029006 shutdown_connections 2023-12-12T09:51:53.336 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.333+0000 7efce6c09640 1 --2- 172.21.15.72:0/3344029006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce00cea70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.336 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.333+0000 7efce6c09640 1 --2- 172.21.15.72:0/3344029006 >> v2:172.21.15.113:3300/0 conn(0x7efce015d380 0x7efce00afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.336 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.333+0000 7efce6c09640 1 -- 172.21.15.72:0/3344029006 >> 172.21.15.72:0/3344029006 conn(0x7efce009c240 msgr2=0x7efce00d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:53.336 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:53.333+0000 7f7236412040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:51:53.339 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.333+0000 7efce6c09640 1 -- 172.21.15.72:0/3344029006 shutdown_connections 2023-12-12T09:51:53.339 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.333+0000 7efce6c09640 1 -- 172.21.15.72:0/3344029006 wait complete. 2023-12-12T09:51:53.339 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.334+0000 7efce6c09640 1 Processor -- start 2023-12-12T09:51:53.339 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.334+0000 7efce6c09640 1 -- start start 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.335+0000 7efce6c09640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce0150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.335+0000 7efce6c09640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efce00d1550 0x7efce00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.335+0000 7efce6c09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce015d380 0x7efce006bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.335+0000 7efce6c09640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c8280 con 0x7efce00d1550 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.335+0000 7efce6c09640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c8560 con 0x7efce015d380 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.335+0000 7efce6c09640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c8840 con 0x7efce00ce630 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efce00d1550 0x7efce00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5c07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce0150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efce00d1550 0x7efce00c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41928/0 (socket says 172.21.15.72:41928) 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 -- 172.21.15.72:0/3142464627 learned_addr learned my addr 172.21.15.72:0/3142464627 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce6408640 1 --2- 172.21.15.72:0/3142464627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce015d380 0x7efce006bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 -- 172.21.15.72:0/3142464627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 msgr2=0x7efce0150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 --2- 172.21.15.72:0/3142464627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce0150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.340 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 -- 172.21.15.72:0/3142464627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce015d380 msgr2=0x7efce006bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 --2- 172.21.15.72:0/3142464627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce015d380 0x7efce006bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 -- 172.21.15.72:0/3142464627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0009400 con 0x7efce00d1550 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.336+0000 7efce5406640 1 --2- 172.21.15.72:0/3142464627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efce00d1550 0x7efce00c9320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efcd0034040 tx=0x7efcd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.337+0000 7efcd67fc640 1 -- 172.21.15.72:0/3142464627 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0032820 con 0x7efce00d1550 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.337+0000 7efcd67fc640 1 -- 172.21.15.72:0/3142464627 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0032980 con 0x7efce00d1550 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.337+0000 7efcd67fc640 1 -- 172.21.15.72:0/3142464627 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0032d30 con 0x7efce00d1550 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.337+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce00c8b20 con 0x7efce00d1550 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.337+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce0150d40 con 0x7efce00d1550 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.337+0000 7efcd67fc640 1 -- 172.21.15.72:0/3142464627 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efcd004e440 con 0x7efce00d1550 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.337+0000 7efcd67fc640 1 -- 172.21.15.72:0/3142464627 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7efcd0031e80 con 0x7efce00d1550 2023-12-12T09:51:53.341 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.338+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce00c9040 con 0x7efce00d1550 2023-12-12T09:51:53.342 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.340+0000 7efcd67fc640 1 -- 172.21.15.72:0/3142464627 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7efce00c9040 con 0x7efce00d1550 2023-12-12T09:51:53.347 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:53.345+0000 7fa3ce413040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:51:53.398 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:53.396+0000 7f7236412040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:51:53.408 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.406+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efce0074450 con 0x7efce00d1550 2023-12-12T09:51:53.408 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:53.406+0000 7fa3ce413040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:51:53.409 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.407+0000 7efcd67fc640 1 -- 172.21.15.72:0/3142464627 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7efcd0005990 con 0x7efce00d1550 2023-12-12T09:51:53.409 INFO:teuthology.misc.health.smithi072.stdout: 2023-12-12T09:51:53.410 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":10,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:51:51.965427+0000","last_up_change":"0.000000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:51:53.410 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efce00d1550 msgr2=0x7efce00c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:53.410 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 --2- 172.21.15.72:0/3142464627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efce00d1550 0x7efce00c9320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efcd0034040 tx=0x7efcd00046c0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.410 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 shutdown_connections 2023-12-12T09:51:53.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 --2- 172.21.15.72:0/3142464627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce015d380 0x7efce006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 --2- 172.21.15.72:0/3142464627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efce00d1550 0x7efce00c9320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 --2- 172.21.15.72:0/3142464627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce00ce630 0x7efce0150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:53.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 >> 172.21.15.72:0/3142464627 conn(0x7efce009c240 msgr2=0x7efce00d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:53.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 shutdown_connections 2023-12-12T09:51:53.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:51:53.408+0000 7efce6c09640 1 -- 172.21.15.72:0/3142464627 wait complete. 2023-12-12T09:51:53.423 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-12T09:51:53.514 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:53.512+0000 7f7236412040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:51:53.521 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-12-12T09:51:53.519+0000 7fa3ce413040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:51:54.558 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:54.556+0000 7f71d730f640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-12T09:51:54.558 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-12-12T09:51:54.556+0000 7f71d730f640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-12T09:51:56.032 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-12-12T09:51:56.031+0000 7fcde50b2500 -1 osd.3 0 log_to_monitors true 2023-12-12T09:51:56.044 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-12-12T09:51:56.041+0000 7fb4abec2500 -1 osd.4 0 log_to_monitors true 2023-12-12T09:51:56.059 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-12-12T09:51:56.057+0000 7f0ff8c13500 -1 osd.5 0 log_to_monitors true 2023-12-12T09:51:56.088 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-12-12T09:51:56.086+0000 7f9feeeaf500 -1 osd.7 0 log_to_monitors true 2023-12-12T09:51:56.268 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-12-12T09:51:56.266+0000 7fa3a294a500 -1 osd.6 0 log_to_monitors true 2023-12-12T09:51:56.279 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-12-12T09:51:56.277+0000 7f5ff8ca7500 -1 osd.1 0 log_to_monitors true 2023-12-12T09:51:56.323 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-12-12T09:51:56.321+0000 7febabca7500 -1 osd.0 0 log_to_monitors true 2023-12-12T09:51:56.329 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-12-12T09:51:56.327+0000 7f2498438500 -1 osd.2 0 log_to_monitors true 2023-12-12T09:51:59.342 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-12-12T09:51:59.340+0000 7f9feae3d640 -1 osd.7 0 waiting for initial osdmap 2023-12-12T09:51:59.353 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-12-12T09:51:59.351+0000 7fb4a8664640 -1 osd.4 0 waiting for initial osdmap 2023-12-12T09:51:59.360 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-12-12T09:51:59.357+0000 7f9fe2c64640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:51:59.366 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-12-12T09:51:59.364+0000 7fb49fc77640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:51:59.367 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-12-12T09:51:59.366+0000 7f0ff4df2640 -1 osd.5 0 waiting for initial osdmap 2023-12-12T09:51:59.371 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-12-12T09:51:59.369+0000 7f0fec405640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:51:59.387 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-12-12T09:51:59.385+0000 7fa39e8da640 -1 osd.6 0 waiting for initial osdmap 2023-12-12T09:51:59.393 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-12-12T09:51:59.391+0000 7fa396701640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:51:59.744 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-12-12T09:51:59.742+0000 7fcde1040640 -1 osd.3 0 waiting for initial osdmap 2023-12-12T09:51:59.755 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-12-12T09:51:59.753+0000 7fcdd8e67640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:51:59.910 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-12-12T09:51:59.909+0000 7f2494bda640 -1 osd.2 0 waiting for initial osdmap 2023-12-12T09:51:59.916 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-12-12T09:51:59.915+0000 7f248c1ed640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:51:59.925 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-12-12T09:51:59.922+0000 7feba7c35640 -1 osd.0 0 waiting for initial osdmap 2023-12-12T09:51:59.941 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-12-12T09:51:59.939+0000 7feb9fa5c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:51:59.993 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-12-12T09:51:59.992+0000 7f5ff4c35640 -1 osd.1 0 waiting for initial osdmap 2023-12-12T09:52:00.000 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-12-12T09:51:59.999+0000 7f5feca5c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:52:00.229 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:00.409 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.407+0000 7fb254874640 1 Processor -- start 2023-12-12T09:52:00.409 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.407+0000 7fb254874640 1 -- start start 2023-12-12T09:52:00.410 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.408+0000 7fb254874640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2500ccbc0 0x7fb25015e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.410 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.408+0000 7fb254874640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb2500cf9d0 0x7fb25015c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.408+0000 7fb254874640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2500cd700 con 0x7fb2500ceac0 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.408+0000 7fb254874640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2500cb330 con 0x7fb2500cf9d0 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.408+0000 7fb254874640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25015e780 con 0x7fb2500ccbc0 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.409+0000 7fb24f59e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb2500cf9d0 0x7fb25015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.409+0000 7fb24f59e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb2500cf9d0 0x7fb25015c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60660/0 (socket says 172.21.15.72:60660) 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.409+0000 7fb24f59e640 1 -- 172.21.15.72:0/2908800631 learned_addr learned my addr 172.21.15.72:0/2908800631 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.409+0000 7fb24f59e640 1 -- 172.21.15.72:0/2908800631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2500ccbc0 msgr2=0x7fb25015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.409+0000 7fb24f59e640 1 --2- 172.21.15.72:0/2908800631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2500ccbc0 0x7fb25015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.409+0000 7fb24f59e640 1 -- 172.21.15.72:0/2908800631 >> v1:172.21.15.113:6789/0 conn(0x7fb2500ceac0 legacy=0x7fb2500cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:00.411 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.409+0000 7fb24f59e640 1 -- 172.21.15.72:0/2908800631 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500cd9e0 con 0x7fb2500cf9d0 2023-12-12T09:52:00.412 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.410+0000 7fb24f59e640 1 --2- 172.21.15.72:0/2908800631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500cf9d0 0x7fb25015c950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb248009e50 tx=0x7fb24802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38eb16d240a30e08 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:00.412 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.410+0000 7fb24e59c640 1 -- 172.21.15.72:0/2908800631 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24802ca70 con 0x7fb2500cf9d0 2023-12-12T09:52:00.412 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.410+0000 7fb24e59c640 1 -- 172.21.15.72:0/2908800631 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24802cbd0 con 0x7fb2500cf9d0 2023-12-12T09:52:00.412 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.410+0000 7fb24e59c640 1 -- 172.21.15.72:0/2908800631 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480343f0 con 0x7fb2500cf9d0 2023-12-12T09:52:00.413 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.411+0000 7fb254874640 1 -- 172.21.15.72:0/2908800631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500cf9d0 msgr2=0x7fb25015c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.413 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.411+0000 7fb254874640 1 --2- 172.21.15.72:0/2908800631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500cf9d0 0x7fb25015c950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb248009e50 tx=0x7fb24802bed0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.413 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.411+0000 7fb254874640 1 -- 172.21.15.72:0/2908800631 shutdown_connections 2023-12-12T09:52:00.413 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.411+0000 7fb254874640 1 --2- 172.21.15.72:0/2908800631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500cf9d0 0x7fb25015c950 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.413 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.411+0000 7fb254874640 1 --2- 172.21.15.72:0/2908800631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2500ccbc0 0x7fb25015e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.413 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.411+0000 7fb254874640 1 -- 172.21.15.72:0/2908800631 >> 172.21.15.72:0/2908800631 conn(0x7fb2500d0ce0 msgr2=0x7fb2500d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:00.413 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.411+0000 7fb254874640 1 -- 172.21.15.72:0/2908800631 shutdown_connections 2023-12-12T09:52:00.413 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.411+0000 7fb254874640 1 -- 172.21.15.72:0/2908800631 wait complete. 2023-12-12T09:52:00.414 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.412+0000 7fb254874640 1 Processor -- start 2023-12-12T09:52:00.414 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.413+0000 7fb254874640 1 -- start start 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.413+0000 7fb254874640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500ccbc0 0x7fb2501f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.413+0000 7fb254874640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2500ceac0 0x7fb2500b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.413+0000 7fb254874640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2500cf9d0 0x7fb2501f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.413+0000 7fb254874640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501f2a60 con 0x7fb2500ccbc0 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.413+0000 7fb254874640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501600c0 con 0x7fb2500cf9d0 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.413+0000 7fb254874640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501603a0 con 0x7fb2500ceac0 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500ccbc0 0x7fb2501f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500ccbc0 0x7fb2501f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60664/0 (socket says 172.21.15.72:60664) 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 -- 172.21.15.72:0/1500304450 learned_addr learned my addr 172.21.15.72:0/1500304450 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:00.416 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24f59e640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2500cf9d0 0x7fb2501f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.417 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 -- 172.21.15.72:0/1500304450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2500ceac0 msgr2=0x7fb2500b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.417 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2500ceac0 0x7fb2500b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.417 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 -- 172.21.15.72:0/1500304450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2500cf9d0 msgr2=0x7fb2501f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.417 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2500cf9d0 0x7fb2501f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.417 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 -- 172.21.15.72:0/1500304450 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb248009b30 con 0x7fb2500ccbc0 2023-12-12T09:52:00.417 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24f59e640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2500cf9d0 0x7fb2501f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:00.417 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.414+0000 7fb24ed9d640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500ccbc0 0x7fb2501f2520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb24000b4a0 tx=0x7fb24000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:00.418 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.415+0000 7fb2477fe640 1 -- 172.21.15.72:0/1500304450 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240010610 con 0x7fb2500ccbc0 2023-12-12T09:52:00.418 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.415+0000 7fb2477fe640 1 -- 172.21.15.72:0/1500304450 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb240010770 con 0x7fb2500ccbc0 2023-12-12T09:52:00.418 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.415+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb250160680 con 0x7fb2500ccbc0 2023-12-12T09:52:00.418 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.415+0000 7fb2477fe640 1 -- 172.21.15.72:0/1500304450 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240010b20 con 0x7fb2500ccbc0 2023-12-12T09:52:00.418 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.415+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2500c8ec0 con 0x7fb2500ccbc0 2023-12-12T09:52:00.418 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.416+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2500673f0 con 0x7fb2500ccbc0 2023-12-12T09:52:00.419 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.417+0000 7fb2477fe640 1 -- 172.21.15.72:0/1500304450 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fb240010c80 con 0x7fb2500ccbc0 2023-12-12T09:52:00.419 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.417+0000 7fb2477fe640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb23006adb0 0x7fb23006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.419 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.418+0000 7fb2477fe640 1 -- 172.21.15.72:0/1500304450 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb240088ca0 con 0x7fb2500ccbc0 2023-12-12T09:52:00.419 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.418+0000 7fb247fff640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb23006adb0 0x7fb23006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.420 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.418+0000 7fb247fff640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb23006adb0 0x7fb23006d270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb25015daf0 tx=0x7fb238007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:00.424 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.422+0000 7fb2477fe640 1 -- 172.21.15.72:0/1500304450 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2400542f0 con 0x7fb2500ccbc0 2023-12-12T09:52:00.608 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.606+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb250066f60 con 0x7fb2500ccbc0 2023-12-12T09:52:00.609 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.607+0000 7fb2477fe640 1 -- 172.21.15.72:0/1500304450 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10908 (secure 0 0 0) 0x7fb240014090 con 0x7fb2500ccbc0 2023-12-12T09:52:00.609 INFO:teuthology.misc.health.smithi072.stdout: 2023-12-12T09:52:00.609 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":13,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:00.357249+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:00.612 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb23006adb0 msgr2=0x7fb23006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.612 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb23006adb0 0x7fb23006d270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb25015daf0 tx=0x7fb238007990 comp rx=0 tx=0).stop 2023-12-12T09:52:00.612 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500ccbc0 msgr2=0x7fb2501f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.612 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500ccbc0 0x7fb2501f2520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb24000b4a0 tx=0x7fb24000b960 comp rx=0 tx=0).stop 2023-12-12T09:52:00.612 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 shutdown_connections 2023-12-12T09:52:00.612 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb23006adb0 0x7fb23006d270 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.612 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2500cf9d0 0x7fb2501f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.613 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2500ceac0 0x7fb2500b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.613 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 --2- 172.21.15.72:0/1500304450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2500ccbc0 0x7fb2501f2520 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.613 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 >> 172.21.15.72:0/1500304450 conn(0x7fb2500d0ce0 msgr2=0x7fb2500cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:00.613 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.610+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 shutdown_connections 2023-12-12T09:52:00.613 INFO:teuthology.misc.health.smithi072.stderr:2023-12-12T09:52:00.611+0000 7fb254874640 1 -- 172.21.15.72:0/1500304450 wait complete. 2023-12-12T09:52:00.625 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-12T09:52:00.625 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-12T09:52:00.626 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-12T09:52:00.626 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-12T09:52:00.630 INFO:tasks.ceph.mds.a:Started 2023-12-12T09:52:00.630 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-12T09:52:00.630 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-12T09:52:00.635 INFO:tasks.ceph.mds.c:Started 2023-12-12T09:52:00.635 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-12T09:52:00.635 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-12T09:52:00.644 INFO:tasks.ceph.mds.b:Started 2023-12-12T09:52:00.644 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-12T09:52:00.644 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-12T09:52:00.653 INFO:tasks.ceph.mds.d:Started 2023-12-12T09:52:00.653 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-12T09:52:00.653 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:00.779 INFO:tasks.ceph.mds.a.smithi072.stdout:starting mds.a at 2023-12-12T09:52:00.781 INFO:tasks.ceph.mds.c.smithi072.stdout:starting mds.c at 2023-12-12T09:52:00.813 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-12-12T09:52:00.814 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-12-12T09:52:00.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.830+0000 7f3e4ffff640 1 Processor -- start 2023-12-12T09:52:00.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.830+0000 7f3e4ffff640 1 -- start start 2023-12-12T09:52:00.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e5015d390 0x7f3e500b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3e500d1b20 0x7f3e500b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4ffff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e500b0a00 con 0x7f3e500d0c10 2023-12-12T09:52:00.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4ffff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e500ceea0 con 0x7f3e500d1b20 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4ffff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5015d770 con 0x7f3e5015d390 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4e7fc640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f3e500d0c10 0x7f3e500d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56044/0 (socket says 172.21.15.72:56044) 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4e7fc640 1 -- 172.21.15.72:0/3407801772 learned_addr learned my addr 172.21.15.72:0/3407801772 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4f7fe640 1 --2- 172.21.15.72:0/3407801772 >> v2:172.21.15.72:3300/0 conn(0x7f3e500d1b20 0x7f3e500b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.831+0000 7f3e4effd640 1 --2- 172.21.15.72:0/3407801772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e5015d390 0x7f3e500b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.832+0000 7f3e4f7fe640 1 -- 172.21.15.72:0/3407801772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e5015d390 msgr2=0x7f3e500b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.832+0000 7f3e4f7fe640 1 --2- 172.21.15.72:0/3407801772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e5015d390 0x7f3e500b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.832+0000 7f3e4f7fe640 1 -- 172.21.15.72:0/3407801772 >> v1:172.21.15.72:6789/0 conn(0x7f3e500d0c10 legacy=0x7f3e500d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.832+0000 7f3e4f7fe640 1 -- 172.21.15.72:0/3407801772 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e500b1840 con 0x7f3e500d1b20 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.832+0000 7f3e4effd640 1 --2- 172.21.15.72:0/3407801772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e5015d390 0x7f3e500b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:00.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.832+0000 7f3e4dffb640 1 -- 172.21.15.72:0/3407801772 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812715748 0 0) 0x7f3e500b0a00 con 0x7f3e500d0c10 2023-12-12T09:52:00.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4f7fe640 1 --2- 172.21.15.72:0/3407801772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d1b20 0x7f3e500b04c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009d90 tx=0x7f3e4802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc76eb989504480 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:00.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4dffb640 1 -- 172.21.15.72:0/3407801772 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4802cbc0 con 0x7f3e500d1b20 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4dffb640 1 -- 172.21.15.72:0/3407801772 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4802cd20 con 0x7f3e500d1b20 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4dffb640 1 -- 172.21.15.72:0/3407801772 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48034660 con 0x7f3e500d1b20 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4ffff640 1 -- 172.21.15.72:0/3407801772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d1b20 msgr2=0x7f3e500b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/3407801772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d1b20 0x7f3e500b04c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009d90 tx=0x7f3e4802be10 comp rx=0 tx=0).stop 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4ffff640 1 -- 172.21.15.72:0/3407801772 shutdown_connections 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/3407801772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d1b20 0x7f3e500b04c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/3407801772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e5015d390 0x7f3e500b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.833+0000 7f3e4ffff640 1 -- 172.21.15.72:0/3407801772 >> 172.21.15.72:0/3407801772 conn(0x7f3e500d2ec0 msgr2=0x7f3e500d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.834+0000 7f3e4ffff640 1 -- 172.21.15.72:0/3407801772 shutdown_connections 2023-12-12T09:52:00.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.834+0000 7f3e4ffff640 1 -- 172.21.15.72:0/3407801772 wait complete. 2023-12-12T09:52:00.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.835+0000 7f3e4ffff640 1 Processor -- start 2023-12-12T09:52:00.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.835+0000 7f3e4ffff640 1 -- start start 2023-12-12T09:52:00.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.836+0000 7f3e4ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d0c10 0x7f3e501f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.836+0000 7f3e4ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e500d1b20 0x7f3e501490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.836+0000 7f3e4ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5015d390 0x7f3e5009f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.836+0000 7f3e4ffff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50149b70 con 0x7f3e500d0c10 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.836+0000 7f3e4ffff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50149e50 con 0x7f3e5015d390 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.836+0000 7f3e4ffff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5014a130 con 0x7f3e500d1b20 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.836+0000 7f3e4effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d0c10 0x7f3e501f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d0c10 0x7f3e501f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60720/0 (socket says 172.21.15.72:60720) 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4effd640 1 -- 172.21.15.72:0/1903623734 learned_addr learned my addr 172.21.15.72:0/1903623734 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4e7fc640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e500d1b20 0x7f3e501490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4f7fe640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5015d390 0x7f3e5009f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4effd640 1 -- 172.21.15.72:0/1903623734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e500d1b20 msgr2=0x7f3e501490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4effd640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e500d1b20 0x7f3e501490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4effd640 1 -- 172.21.15.72:0/1903623734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5015d390 msgr2=0x7f3e5009f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4effd640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5015d390 0x7f3e5009f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4effd640 1 -- 172.21.15.72:0/1903623734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48009a70 con 0x7f3e500d0c10 2023-12-12T09:52:00.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4effd640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d0c10 0x7f3e501f2630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000d7d0 tx=0x7f3e4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:00.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e377fe640 1 -- 172.21.15.72:0/1903623734 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004030 con 0x7f3e500d0c10 2023-12-12T09:52:00.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e377fe640 1 -- 172.21.15.72:0/1903623734 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40004190 con 0x7f3e500d0c10 2023-12-12T09:52:00.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e377fe640 1 -- 172.21.15.72:0/1903623734 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004500 con 0x7f3e500d0c10 2023-12-12T09:52:00.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.837+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5014a410 con 0x7f3e500d0c10 2023-12-12T09:52:00.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.838+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5015c450 con 0x7f3e500d0c10 2023-12-12T09:52:00.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.839+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e500b0ef0 con 0x7f3e500d0c10 2023-12-12T09:52:00.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.840+0000 7f3e377fe640 1 -- 172.21.15.72:0/1903623734 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f3e4000b6c0 con 0x7f3e500d0c10 2023-12-12T09:52:00.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.840+0000 7f3e377fe640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3e3c06adb0 0x7f3e3c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.840+0000 7f3e377fe640 1 -- 172.21.15.72:0/1903623734 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f3e40089040 con 0x7f3e500d0c10 2023-12-12T09:52:00.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.841+0000 7f3e4e7fc640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3e3c06adb0 0x7f3e3c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.841+0000 7f3e4e7fc640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3e3c06adb0 0x7f3e3c06d270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e44000f60 tx=0x7f3e44007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:00.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:00.845+0000 7f3e377fe640 1 -- 172.21.15.72:0/1903623734 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e500b0ef0 con 0x7f3e500d0c10 2023-12-12T09:52:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.022+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e500b0ef0 con 0x7f3e500d0c10 2023-12-12T09:52:01.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.024+0000 7f3e377fe640 1 -- 172.21.15.72:0/1903623734 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10908 (secure 0 0 0) 0x7f3e500b0ef0 con 0x7f3e500d0c10 2023-12-12T09:52:01.026 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:01.026 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":13,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:00.357249+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3e3c06adb0 msgr2=0x7f3e3c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3e3c06adb0 0x7f3e3c06d270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e44000f60 tx=0x7f3e44007d50 comp rx=0 tx=0).stop 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d0c10 msgr2=0x7f3e501f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d0c10 0x7f3e501f2630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000d7d0 tx=0x7f3e4000dc90 comp rx=0 tx=0).stop 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 shutdown_connections 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3e3c06adb0 0x7f3e3c06d270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5015d390 0x7f3e5009f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e500d1b20 0x7f3e501490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 --2- 172.21.15.72:0/1903623734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e500d0c10 0x7f3e501f2630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 >> 172.21.15.72:0/1903623734 conn(0x7f3e500d2ec0 msgr2=0x7f3e500d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 shutdown_connections 2023-12-12T09:52:01.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.027+0000 7f3e4ffff640 1 -- 172.21.15.72:0/1903623734 wait complete. 2023-12-12T09:52:01.046 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-12-12T09:52:01.046 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:52:01.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.235+0000 7f4e7eb25640 1 Processor -- start 2023-12-12T09:52:01.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.235+0000 7f4e7eb25640 1 -- start start 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e7eb25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e780cd000 0x7f4e7806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e7eb25640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4e78146ba0 0x7f4e7806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e7eb25640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7806d260 con 0x7f4e781467c0 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e7eb25640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e780ac860 con 0x7f4e78146ba0 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e7eb25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7806d510 con 0x7f4e780cd000 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e7d0c2640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4e78146ba0 0x7f4e7806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e7d0c2640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4e78146ba0 0x7f4e7806cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60728/0 (socket says 172.21.15.72:60728) 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e7d0c2640 1 -- 172.21.15.72:0/86580080 learned_addr learned my addr 172.21.15.72:0/86580080 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:01.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.237+0000 7f4e77fff640 1 --2- 172.21.15.72:0/86580080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e780cd000 0x7f4e7806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:01.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.238+0000 7f4e7d0c2640 1 -- 172.21.15.72:0/86580080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e780cd000 msgr2=0x7f4e7806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:01.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.238+0000 7f4e7d0c2640 1 --2- 172.21.15.72:0/86580080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e780cd000 0x7f4e7806c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.238+0000 7f4e7d0c2640 1 -- 172.21.15.72:0/86580080 >> v1:172.21.15.113:6789/0 conn(0x7f4e781467c0 legacy=0x7f4e780cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:01.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.238+0000 7f4e7d0c2640 1 -- 172.21.15.72:0/86580080 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7806d7f0 con 0x7f4e78146ba0 2023-12-12T09:52:01.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.239+0000 7f4e7d0c2640 1 --2- 172.21.15.72:0/86580080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e78146ba0 0x7f4e7806cd20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009e50 tx=0x7f4e7002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8bc308e334c015 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:01.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.239+0000 7f4e777fe640 1 -- 172.21.15.72:0/86580080 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7002cb00 con 0x7f4e78146ba0 2023-12-12T09:52:01.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.239+0000 7f4e777fe640 1 -- 172.21.15.72:0/86580080 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7002cc60 con 0x7f4e78146ba0 2023-12-12T09:52:01.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.239+0000 7f4e777fe640 1 -- 172.21.15.72:0/86580080 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e700343f0 con 0x7f4e78146ba0 2023-12-12T09:52:01.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.240+0000 7f4e7eb25640 1 -- 172.21.15.72:0/86580080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e78146ba0 msgr2=0x7f4e7806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:01.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.240+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/86580080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e78146ba0 0x7f4e7806cd20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009e50 tx=0x7f4e7002bf60 comp rx=0 tx=0).stop 2023-12-12T09:52:01.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.240+0000 7f4e7eb25640 1 -- 172.21.15.72:0/86580080 shutdown_connections 2023-12-12T09:52:01.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.240+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/86580080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e78146ba0 0x7f4e7806cd20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.240+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/86580080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e780cd000 0x7f4e7806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.240+0000 7f4e7eb25640 1 -- 172.21.15.72:0/86580080 >> 172.21.15.72:0/86580080 conn(0x7f4e780b0a90 msgr2=0x7f4e780b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:01.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.240+0000 7f4e7eb25640 1 -- 172.21.15.72:0/86580080 shutdown_connections 2023-12-12T09:52:01.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.240+0000 7f4e7eb25640 1 -- 172.21.15.72:0/86580080 wait complete. 2023-12-12T09:52:01.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.241+0000 7f4e7eb25640 1 Processor -- start 2023-12-12T09:52:01.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.242+0000 7f4e7eb25640 1 -- start start 2023-12-12T09:52:01.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.242+0000 7f4e7eb25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e780cd000 0x7f4e781f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:01.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.242+0000 7f4e7eb25640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e781467c0 0x7f4e781e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:01.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.242+0000 7f4e7eb25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e78146ba0 0x7f4e78142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.242+0000 7f4e7eb25640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780b4440 con 0x7f4e781467c0 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.242+0000 7f4e7eb25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781f31e0 con 0x7f4e780cd000 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.242+0000 7f4e7eb25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781e7de0 con 0x7f4e78146ba0 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e781467c0 0x7f4e781e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e781467c0 0x7f4e781e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60736/0 (socket says 172.21.15.72:60736) 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 -- 172.21.15.72:0/1000824322 learned_addr learned my addr 172.21.15.72:0/1000824322 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e7d0c2640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e78146ba0 0x7f4e78142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e7c8c1640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e780cd000 0x7f4e781f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 -- 172.21.15.72:0/1000824322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e78146ba0 msgr2=0x7f4e78142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e78146ba0 0x7f4e78142700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 -- 172.21.15.72:0/1000824322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e780cd000 msgr2=0x7f4e781f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e780cd000 0x7f4e781f2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 -- 172.21.15.72:0/1000824322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70009b30 con 0x7f4e781467c0 2023-12-12T09:52:01.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.243+0000 7f4e77fff640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e781467c0 0x7f4e781e78a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400d800 tx=0x7f4e6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:01.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.244+0000 7f4e757fa640 1 -- 172.21.15.72:0/1000824322 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400b6c0 con 0x7f4e781467c0 2023-12-12T09:52:01.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.244+0000 7f4e757fa640 1 -- 172.21.15.72:0/1000824322 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6400b820 con 0x7f4e781467c0 2023-12-12T09:52:01.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.244+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e780b3e80 con 0x7f4e781467c0 2023-12-12T09:52:01.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.244+0000 7f4e757fa640 1 -- 172.21.15.72:0/1000824322 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400bbb0 con 0x7f4e781467c0 2023-12-12T09:52:01.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.244+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e781f3440 con 0x7f4e781467c0 2023-12-12T09:52:01.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.245+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e780745d0 con 0x7f4e781467c0 2023-12-12T09:52:01.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.246+0000 7f4e757fa640 1 -- 172.21.15.72:0/1000824322 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f4e6400bd10 con 0x7f4e781467c0 2023-12-12T09:52:01.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.246+0000 7f4e757fa640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4e5c06adb0 0x7f4e5c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:01.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.247+0000 7f4e757fa640 1 -- 172.21.15.72:0/1000824322 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f4e64089640 con 0x7f4e781467c0 2023-12-12T09:52:01.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.247+0000 7f4e7c8c1640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4e5c06adb0 0x7f4e5c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:01.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.247+0000 7f4e7c8c1640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4e5c06adb0 0x7f4e5c06d270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c000f60 tx=0x7f4e6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:01.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.251+0000 7f4e757fa640 1 -- 172.21.15.72:0/1000824322 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e64054c00 con 0x7f4e781467c0 2023-12-12T09:52:01.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.461+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e780726a0 con 0x7f4e781467c0 2023-12-12T09:52:01.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.462+0000 7f4e757fa640 1 -- 172.21.15.72:0/1000824322 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f4e64058220 con 0x7f4e781467c0 2023-12-12T09:52:01.464 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:01.464 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-12T09:52:01.464 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 2 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4e5c06adb0 msgr2=0x7f4e5c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4e5c06adb0 0x7f4e5c06d270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c000f60 tx=0x7f4e6c009040 comp rx=0 tx=0).stop 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e781467c0 msgr2=0x7f4e781e78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e781467c0 0x7f4e781e78a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400d800 tx=0x7f4e6400dcc0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 shutdown_connections 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4e5c06adb0 0x7f4e5c06d270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e78146ba0 0x7f4e78142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e781467c0 0x7f4e781e78a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 --2- 172.21.15.72:0/1000824322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e780cd000 0x7f4e781f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:01.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 >> 172.21.15.72:0/1000824322 conn(0x7f4e780b0a90 msgr2=0x7f4e78070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:01.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 shutdown_connections 2023-12-12T09:52:01.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:01.464+0000 7f4e7eb25640 1 -- 172.21.15.72:0/1000824322 wait complete. 2023-12-12T09:52:02.695 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:52:02.704 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:52:03.485 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:52:03.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.676+0000 7f666373c640 1 Processor -- start 2023-12-12T09:52:03.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.676+0000 7f666373c640 1 -- start start 2023-12-12T09:52:03.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.678+0000 7f666373c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c157a60 0x7f665c06c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:03.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.678+0000 7f666373c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c06ca40 con 0x7f665c06de20 2023-12-12T09:52:03.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.678+0000 7f666373c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c071cd0 con 0x7f665c15e2b0 2023-12-12T09:52:03.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.678+0000 7f666373c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c070f10 con 0x7f665c157a60 2023-12-12T09:52:03.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.678+0000 7f6660cd7640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f665c06de20 0x7f665c15dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56052/0 (socket says 172.21.15.72:56052) 2023-12-12T09:52:03.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.678+0000 7f6660cd7640 1 -- 172.21.15.72:0/2104922221 learned_addr learned my addr 172.21.15.72:0/2104922221 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f66614d8640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c157a60 0x7f665c06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440913934 0 0) 0x7f665c06ca40 con 0x7f665c06de20 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6648005730 con 0x7f665c06de20 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603144021 0 0) 0x7f6648005730 con 0x7f665c06de20 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c157a60 msgr2=0x7f665c06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c157a60 0x7f665c06c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 >> v1:172.21.15.113:6789/0 conn(0x7f665c15e2b0 legacy=0x7f665c0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0711f0 con 0x7f665c06de20 2023-12-12T09:52:03.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f664c001670 con 0x7f665c06de20 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 >> v1:172.21.15.72:6789/0 conn(0x7f665c06de20 legacy=0x7f665c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648005bb0 0x7f6648005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c157a60 0x7f66480069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6648006f10 0x7f6648007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c070f10 con 0x7f6648006f10 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664800dcc0 con 0x7f6648005bb0 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.679+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664800de80 con 0x7f665c157a60 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.680+0000 7f6661cd9640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6648006f10 0x7f6648007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.680+0000 7f66614d8640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648005bb0 0x7f6648005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.680+0000 7f6660cd7640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c157a60 0x7f66480069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.680+0000 7f6661cd9640 1 -- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c157a60 msgr2=0x7f66480069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.680+0000 7f6661cd9640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c157a60 0x7f66480069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.680+0000 7f6661cd9640 1 -- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648005bb0 msgr2=0x7f6648005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.680+0000 7f6661cd9640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648005bb0 0x7f6648005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.680+0000 7f6661cd9640 1 -- 172.21.15.72:0/2104922221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6648005730 con 0x7f6648006f10 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.681+0000 7f6661cd9640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6648006f10 0x7f6648007390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f665c15ebb0 tx=0x7f6648007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3b193690fd8312 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.681+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654003b70 con 0x7f6648006f10 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.681+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654003cd0 con 0x7f6648006f10 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.681+0000 7f6653fff640 1 -- 172.21.15.72:0/2104922221 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654005310 con 0x7f6648006f10 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.682+0000 7f666373c640 1 -- 172.21.15.72:0/2104922221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6648006f10 msgr2=0x7f6648007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.682+0000 7f666373c640 1 --2- 172.21.15.72:0/2104922221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6648006f10 0x7f6648007390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f665c15ebb0 tx=0x7f6648007a00 comp rx=0 tx=0).stop 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.682+0000 7f6661cd9640 1 -- 172.21.15.72:0/2104922221 reap_dead start 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.682+0000 7f666373c640 1 -- 172.21.15.72:0/2104922221 shutdown_connections 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.682+0000 7f666373c640 1 -- 172.21.15.72:0/2104922221 >> 172.21.15.72:0/2104922221 conn(0x7f665c0b0a90 msgr2=0x7f665c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.682+0000 7f666373c640 1 -- 172.21.15.72:0/2104922221 shutdown_connections 2023-12-12T09:52:03.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.682+0000 7f666373c640 1 -- 172.21.15.72:0/2104922221 wait complete. 2023-12-12T09:52:03.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.684+0000 7f666373c640 1 Processor -- start 2023-12-12T09:52:03.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.685+0000 7f666373c640 1 -- start start 2023-12-12T09:52:03.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f666373c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648006f10 0x7f665c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f666373c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c1ee880 0x7f665c145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f666373c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f665c0b2870 0x7f665c15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f666373c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0b50f0 con 0x7f665c0b2870 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f666373c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c145c40 con 0x7f6648006f10 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f666373c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0b47c0 con 0x7f665c1ee880 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6661cd9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f665c0b2870 0x7f665c15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6661cd9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f665c0b2870 0x7f665c15a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60766/0 (socket says 172.21.15.72:60766) 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6661cd9640 1 -- 172.21.15.72:0/1417575658 learned_addr learned my addr 172.21.15.72:0/1417575658 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f66614d8640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648006f10 0x7f665c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6660cd7640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c1ee880 0x7f665c145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6661cd9640 1 -- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c1ee880 msgr2=0x7f665c145700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6661cd9640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c1ee880 0x7f665c145700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6661cd9640 1 -- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648006f10 msgr2=0x7f665c15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:03.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6661cd9640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648006f10 0x7f665c15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f6661cd9640 1 -- 172.21.15.72:0/1417575658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0b4aa0 con 0x7f665c0b2870 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.686+0000 7f66614d8640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648006f10 0x7f665c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.687+0000 7f6661cd9640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f665c0b2870 0x7f665c15a520 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6654001ee0 tx=0x7f66540014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.687+0000 7f6651ffb640 1 -- 172.21.15.72:0/1417575658 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665403e070 con 0x7f665c0b2870 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.687+0000 7f6651ffb640 1 -- 172.21.15.72:0/1417575658 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654003e40 con 0x7f665c0b2870 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.687+0000 7f6651ffb640 1 -- 172.21.15.72:0/1417575658 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654006d50 con 0x7f665c0b2870 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.687+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c0b4d80 con 0x7f665c0b2870 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.687+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c0b2d00 con 0x7f665c0b2870 2023-12-12T09:52:03.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.688+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c15b730 con 0x7f665c0b2870 2023-12-12T09:52:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.689+0000 7f6651ffb640 1 -- 172.21.15.72:0/1417575658 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f6654041070 con 0x7f665c0b2870 2023-12-12T09:52:03.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.691+0000 7f6651ffb640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f664006ade0 0x7f664006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:03.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.691+0000 7f6651ffb640 1 -- 172.21.15.72:0/1417575658 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66540aa090 con 0x7f665c0b2870 2023-12-12T09:52:03.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.691+0000 7f66614d8640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f664006ade0 0x7f664006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:03.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.692+0000 7f66614d8640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f664006ade0 0x7f664006d2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6658009a00 tx=0x7f6658008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:03.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.696+0000 7f6651ffb640 1 -- 172.21.15.72:0/1417575658 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6654075550 con 0x7f665c0b2870 2023-12-12T09:52:03.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.937+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f665c0745d0 con 0x7f665c0b2870 2023-12-12T09:52:03.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.938+0000 7f6651ffb640 1 -- 172.21.15.72:0/1417575658 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f6654048090 con 0x7f665c0b2870 2023-12-12T09:52:03.940 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:03.940 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-12T09:52:03.940 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 3 2023-12-12T09:52:03.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f664006ade0 msgr2=0x7f664006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:03.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f664006ade0 0x7f664006d2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6658009a00 tx=0x7f6658008040 comp rx=0 tx=0).stop 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f665c0b2870 msgr2=0x7f665c15a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f665c0b2870 0x7f665c15a520 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6654001ee0 tx=0x7f66540014a0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 shutdown_connections 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f664006ade0 0x7f664006d2a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f665c0b2870 0x7f665c15a520 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f665c1ee880 0x7f665c145700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.940+0000 7f666373c640 1 --2- 172.21.15.72:0/1417575658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6648006f10 0x7f665c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.941+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 >> 172.21.15.72:0/1417575658 conn(0x7f665c0b0a90 msgr2=0x7f665c0700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.941+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 shutdown_connections 2023-12-12T09:52:03.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:03.941+0000 7f666373c640 1 -- 172.21.15.72:0/1417575658 wait complete. 2023-12-12T09:52:03.963 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.140+0000 7f8d1dbc9640 1 Processor -- start 2023-12-12T09:52:04.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.140+0000 7f8d1dbc9640 1 -- start start 2023-12-12T09:52:04.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d1dbc9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d18143e00 0x7f8d180b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d1dbc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 0x7f8d1815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d1dbc9640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d180b3bb0 con 0x7f8d18072180 2023-12-12T09:52:04.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d1dbc9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d181441e0 con 0x7f8d18143e00 2023-12-12T09:52:04.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d1dbc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18072680 con 0x7f8d1806c410 2023-12-12T09:52:04.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d0effd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f8d18072180 0x7f8d180b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56064/0 (socket says 172.21.15.72:56064) 2023-12-12T09:52:04.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d0effd640 1 -- 172.21.15.72:0/82293894 learned_addr learned my addr 172.21.15.72:0/82293894 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:04.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d177fe640 1 --2- 172.21.15.72:0/82293894 >> v2:172.21.15.113:3300/0 conn(0x7f8d18143e00 0x7f8d180b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d17fff640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 0x7f8d1815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57571935 0 0) 0x7f8d180b3bb0 con 0x7f8d18072180 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.142+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d04003520 con 0x7f8d18072180 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901377694 0 0) 0x7f8d04003520 con 0x7f8d18072180 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 msgr2=0x7f8d1815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 0x7f8d1815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 >> v2:172.21.15.113:3300/0 conn(0x7f8d18143e00 msgr2=0x7f8d180b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 --2- 172.21.15.72:0/82293894 >> v2:172.21.15.113:3300/0 conn(0x7f8d18143e00 0x7f8d180b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180b1e30 con 0x7f8d18072180 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f8d00001670 con 0x7f8d18072180 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d17fff640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 0x7f8d1815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d177fe640 1 --2- 172.21.15.72:0/82293894 >> v2:172.21.15.113:3300/0 conn(0x7f8d18143e00 0x7f8d180b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 >> v1:172.21.15.72:6789/0 conn(0x7f8d18072180 legacy=0x7f8d180b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d04003970 0x7f8d04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d040047c0 0x7f8d04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 0x7f8d04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d181441e0 con 0x7f8d04003970 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18072680 con 0x7f8d040047c0 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0400ba00 con 0x7f8d1806c410 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d177fe640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d04003970 0x7f8d04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.143+0000 7f8d17fff640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 0x7f8d04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.144+0000 7f8d0effd640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d040047c0 0x7f8d04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.144+0000 7f8d177fe640 1 -- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 msgr2=0x7f8d04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.144+0000 7f8d177fe640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d1806c410 0x7f8d04005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.144+0000 7f8d177fe640 1 -- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d040047c0 msgr2=0x7f8d04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.144+0000 7f8d177fe640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d040047c0 0x7f8d04004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.144+0000 7f8d177fe640 1 -- 172.21.15.72:0/82293894 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04003520 con 0x7f8d04003970 2023-12-12T09:52:04.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.145+0000 7f8d177fe640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d04003970 0x7f8d04003d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d080073e0 tx=0x7f8d040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2196ca9ddf6d8f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.145+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0802cdc0 con 0x7f8d04003970 2023-12-12T09:52:04.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.145+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d08034440 con 0x7f8d04003970 2023-12-12T09:52:04.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.145+0000 7f8d16ffd640 1 -- 172.21.15.72:0/82293894 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d080347b0 con 0x7f8d04003970 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.146+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/82293894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d04003970 msgr2=0x7f8d04003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.146+0000 7f8d1dbc9640 1 --2- 172.21.15.72:0/82293894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d04003970 0x7f8d04003d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d080073e0 tx=0x7f8d040057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.146+0000 7f8d17fff640 1 -- 172.21.15.72:0/82293894 reap_dead start 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.146+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/82293894 shutdown_connections 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.146+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/82293894 >> 172.21.15.72:0/82293894 conn(0x7f8d180b0a90 msgr2=0x7f8d180b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.146+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/82293894 shutdown_connections 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.146+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/82293894 wait complete. 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.148+0000 7f8d1dbc9640 1 Processor -- start 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.148+0000 7f8d1dbc9640 1 -- start start 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d1dbc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d04003970 0x7f8d181423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d1dbc9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d18142910 0x7f8d18137690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d1dbc9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d18137bd0 0x7f8d18135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d1dbc9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1813fb70 con 0x7f8d18137bd0 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d1dbc9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18138050 con 0x7f8d18142910 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d1dbc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18142e20 con 0x7f8d04003970 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d18137bd0 0x7f8d18135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d18137bd0 0x7f8d18135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60786/0 (socket says 172.21.15.72:60786) 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d17fff640 1 -- 172.21.15.72:0/1100644004 learned_addr learned my addr 172.21.15.72:0/1100644004 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d17fff640 1 -- 172.21.15.72:0/1100644004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d04003970 msgr2=0x7f8d181423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d17fff640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d04003970 0x7f8d181423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d17fff640 1 -- 172.21.15.72:0/1100644004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d18142910 msgr2=0x7f8d18137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d0effd640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d18142910 0x7f8d18137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d17fff640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d18142910 0x7f8d18137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.149+0000 7f8d17fff640 1 -- 172.21.15.72:0/1100644004 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d18138a90 con 0x7f8d18137bd0 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.150+0000 7f8d17fff640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d18137bd0 0x7f8d18135510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d180741b0 tx=0x7f8d100108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.150+0000 7f8d14ff9640 1 -- 172.21.15.72:0/1100644004 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10004440 con 0x7f8d18137bd0 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.150+0000 7f8d14ff9640 1 -- 172.21.15.72:0/1100644004 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1000c050 con 0x7f8d18137bd0 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.150+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d18138d70 con 0x7f8d18137bd0 2023-12-12T09:52:04.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.150+0000 7f8d14ff9640 1 -- 172.21.15.72:0/1100644004 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10003850 con 0x7f8d18137bd0 2023-12-12T09:52:04.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.150+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d18139330 con 0x7f8d18137bd0 2023-12-12T09:52:04.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.151+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d180745d0 con 0x7f8d18137bd0 2023-12-12T09:52:04.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.152+0000 7f8d14ff9640 1 -- 172.21.15.72:0/1100644004 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f8d100039b0 con 0x7f8d18137bd0 2023-12-12T09:52:04.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.153+0000 7f8d14ff9640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8cf806adb0 0x7f8cf806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.153+0000 7f8d14ff9640 1 -- 172.21.15.72:0/1100644004 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d10088ef0 con 0x7f8d18137bd0 2023-12-12T09:52:04.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.153+0000 7f8d177fe640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8cf806adb0 0x7f8cf806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.154+0000 7f8d177fe640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8cf806adb0 0x7f8cf806d270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d080073e0 tx=0x7f8d0802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.157+0000 7f8d14ff9640 1 -- 172.21.15.72:0/1100644004 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d100543d0 con 0x7f8d18137bd0 2023-12-12T09:52:04.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.347+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d1815afc0 con 0x7f8d18137bd0 2023-12-12T09:52:04.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.349+0000 7f8d14ff9640 1 -- 172.21.15.72:0/1100644004 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f8d10000e60 con 0x7f8d18137bd0 2023-12-12T09:52:04.351 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:04.351 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:03.815466+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:04.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.351+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8cf806adb0 msgr2=0x7f8cf806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.351+0000 7f8d1dbc9640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8cf806adb0 0x7f8cf806d270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d080073e0 tx=0x7f8d0802e000 comp rx=0 tx=0).stop 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.351+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d18137bd0 msgr2=0x7f8d18135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.351+0000 7f8d1dbc9640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d18137bd0 0x7f8d18135510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d180741b0 tx=0x7f8d100108b0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.351+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 shutdown_connections 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.352+0000 7f8d1dbc9640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8cf806adb0 0x7f8cf806d270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.352+0000 7f8d1dbc9640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d18137bd0 0x7f8d18135510 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.352+0000 7f8d1dbc9640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d18142910 0x7f8d18137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.352+0000 7f8d1dbc9640 1 --2- 172.21.15.72:0/1100644004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d04003970 0x7f8d181423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.352+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 >> 172.21.15.72:0/1100644004 conn(0x7f8d180b0a90 msgr2=0x7f8d18070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.352+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 shutdown_connections 2023-12-12T09:52:04.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.352+0000 7f8d1dbc9640 1 -- 172.21.15.72:0/1100644004 wait complete. 2023-12-12T09:52:04.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:52:04.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:52:04.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.561+0000 7f307a9c9640 1 Processor -- start 2023-12-12T09:52:04.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.562+0000 7f307a9c9640 1 -- start start 2023-12-12T09:52:04.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.563+0000 7f307a9c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 0x7f307415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.563+0000 7f307a9c9640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740ce6e0 con 0x7f307415e720 2023-12-12T09:52:04.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.563+0000 7f307a9c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740ce9c0 con 0x7f30740b4550 2023-12-12T09:52:04.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.563+0000 7f307a9c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30740d21f0 con 0x7f30740d0140 2023-12-12T09:52:04.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.563+0000 7f306bfff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f307415e720 0x7f30740cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56070/0 (socket says 172.21.15.72:56070) 2023-12-12T09:52:04.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.563+0000 7f306bfff640 1 -- 172.21.15.72:0/1897603300 learned_addr learned my addr 172.21.15.72:0/1897603300 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:04.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f3078f66640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 0x7f307415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261821823 0 0) 0x7f30740ce6e0 con 0x7f307415e720 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3060003520 con 0x7f307415e720 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1839281828 0 0) 0x7f3060003520 con 0x7f307415e720 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 msgr2=0x7f307415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 0x7f307415cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 >> v1:172.21.15.113:6789/0 conn(0x7f30740b4550 legacy=0x7f30740cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740ceca0 con 0x7f307415e720 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f305c001670 con 0x7f307415e720 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 >> v1:172.21.15.72:6789/0 conn(0x7f307415e720 legacy=0x7f30740cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30600039a0 0x7f3060003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 0x7f30600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3060004d00 0x7f3060005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740d21f0 con 0x7f3060004d00 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306000ba90 con 0x7f30600039a0 2023-12-12T09:52:04.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.564+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306000bc50 con 0x7f30740d0140 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f3078f66640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3060004d00 0x7f3060005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f306bfff640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 0x7f30600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f3073fff640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30600039a0 0x7f3060003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f3078f66640 1 -- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 msgr2=0x7f30600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f3078f66640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 0x7f30600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f3078f66640 1 -- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30600039a0 msgr2=0x7f3060003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f3078f66640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30600039a0 0x7f3060003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f3078f66640 1 -- 172.21.15.72:0/1897603300 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060003520 con 0x7f3060004d00 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.565+0000 7f306bfff640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30740d0140 0x7f30600047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.566+0000 7f3078f66640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3060004d00 0x7f3060005180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f306c009f90 tx=0x7f30600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cd709475ffdaba server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.566+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c02cbd0 con 0x7f3060004d00 2023-12-12T09:52:04.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.566+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c00f040 con 0x7f3060004d00 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.566+0000 7f30737fe640 1 -- 172.21.15.72:0/1897603300 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c034440 con 0x7f3060004d00 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.567+0000 7f307a9c9640 1 -- 172.21.15.72:0/1897603300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3060004d00 msgr2=0x7f3060005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.567+0000 7f307a9c9640 1 --2- 172.21.15.72:0/1897603300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3060004d00 0x7f3060005180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f306c009f90 tx=0x7f30600057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.567+0000 7f3078f66640 1 -- 172.21.15.72:0/1897603300 reap_dead start 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.567+0000 7f307a9c9640 1 -- 172.21.15.72:0/1897603300 shutdown_connections 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.567+0000 7f307a9c9640 1 -- 172.21.15.72:0/1897603300 >> 172.21.15.72:0/1897603300 conn(0x7f307409c430 msgr2=0x7f30740d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.567+0000 7f307a9c9640 1 -- 172.21.15.72:0/1897603300 shutdown_connections 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.567+0000 7f307a9c9640 1 -- 172.21.15.72:0/1897603300 wait complete. 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.569+0000 7f307a9c9640 1 Processor -- start 2023-12-12T09:52:04.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.569+0000 7f307a9c9640 1 -- start start 2023-12-12T09:52:04.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f307a9c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3060004d00 0x7f3074156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f307a9c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30741e98d0 0x7f30740b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f307a9c9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30740c9000 0x7f30740c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f307a9c9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740b5d10 con 0x7f30740c9000 2023-12-12T09:52:04.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f307a9c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740c7440 con 0x7f3060004d00 2023-12-12T09:52:04.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f307a9c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30740c7720 con 0x7f30741e98d0 2023-12-12T09:52:04.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3078f66640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30740c9000 0x7f30740c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3078f66640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30740c9000 0x7f30740c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60806/0 (socket says 172.21.15.72:60806) 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3078f66640 1 -- 172.21.15.72:0/3496194294 learned_addr learned my addr 172.21.15.72:0/3496194294 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3073fff640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3060004d00 0x7f3074156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3078f66640 1 -- 172.21.15.72:0/3496194294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30741e98d0 msgr2=0x7f30740b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3078f66640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30741e98d0 0x7f30740b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3078f66640 1 -- 172.21.15.72:0/3496194294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3060004d00 msgr2=0x7f3074156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3078f66640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3060004d00 0x7f3074156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3078f66640 1 -- 172.21.15.72:0/3496194294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740c7a00 con 0x7f30740c9000 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.570+0000 7f3073fff640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3060004d00 0x7f3074156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.571+0000 7f3078f66640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30740c9000 0x7f30740c9480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f306c037040 tx=0x7f306c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.571+0000 7f30717fa640 1 -- 172.21.15.72:0/3496194294 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c034d50 con 0x7f30740c9000 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.571+0000 7f30717fa640 1 -- 172.21.15.72:0/3496194294 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c003990 con 0x7f30740c9000 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.571+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30740cb340 con 0x7f30740c9000 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.571+0000 7f30717fa640 1 -- 172.21.15.72:0/3496194294 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c003d20 con 0x7f30740c9000 2023-12-12T09:52:04.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.571+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30740c99c0 con 0x7f30740c9000 2023-12-12T09:52:04.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.572+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30740cb860 con 0x7f30740c9000 2023-12-12T09:52:04.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.573+0000 7f30717fa640 1 -- 172.21.15.72:0/3496194294 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f306c004030 con 0x7f30740c9000 2023-12-12T09:52:04.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.574+0000 7f30717fa640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f305406ae40 0x7f305406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.574+0000 7f30717fa640 1 -- 172.21.15.72:0/3496194294 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f306c0aab50 con 0x7f30740c9000 2023-12-12T09:52:04.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.574+0000 7f3073fff640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f305406ae40 0x7f305406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.575+0000 7f3073fff640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f305406ae40 0x7f305406d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3064001630 tx=0x7f3064009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.579+0000 7f30717fa640 1 -- 172.21.15.72:0/3496194294 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f30740cb860 con 0x7f30740c9000 2023-12-12T09:52:04.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.750+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30740cb860 con 0x7f30740c9000 2023-12-12T09:52:04.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.751+0000 7f30717fa640 1 -- 172.21.15.72:0/3496194294 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30740cb860 con 0x7f30740c9000 2023-12-12T09:52:04.753 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:52:04.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.753+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f305406ae40 msgr2=0x7f305406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.753+0000 7f307a9c9640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f305406ae40 0x7f305406d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3064001630 tx=0x7f3064009040 comp rx=0 tx=0).stop 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.753+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30740c9000 msgr2=0x7f30740c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.753+0000 7f307a9c9640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30740c9000 0x7f30740c9480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f306c037040 tx=0x7f306c009750 comp rx=0 tx=0).stop 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.754+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 shutdown_connections 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.754+0000 7f307a9c9640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f305406ae40 0x7f305406d300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.754+0000 7f307a9c9640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30740c9000 0x7f30740c9480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.754+0000 7f307a9c9640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30741e98d0 0x7f30740b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.754+0000 7f307a9c9640 1 --2- 172.21.15.72:0/3496194294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3060004d00 0x7f3074156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.754+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 >> 172.21.15.72:0/3496194294 conn(0x7f307409c430 msgr2=0x7f30740cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.754+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 shutdown_connections 2023-12-12T09:52:04.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.754+0000 7f307a9c9640 1 -- 172.21.15.72:0/3496194294 wait complete. 2023-12-12T09:52:04.772 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-12T09:52:04.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.971+0000 7fe41109f640 1 Processor -- start 2023-12-12T09:52:04.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.972+0000 7fe41109f640 1 -- start start 2023-12-12T09:52:04.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.973+0000 7fe41109f640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe40c0c7170 0x7fe40c0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.973+0000 7fe41109f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c0c75f0 0x7fe40c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.973+0000 7fe41109f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c0d3a30 con 0x7fe40c160810 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.973+0000 7fe41109f640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40c0cef40 con 0x7fe40c0c7170 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.973+0000 7fe41109f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0b3d10 con 0x7fe40c0c75f0 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.973+0000 7fe40b7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe40c0c7170 0x7fe40c0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.973+0000 7fe40b7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe40c0c7170 0x7fe40c0cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60818/0 (socket says 172.21.15.72:60818) 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.973+0000 7fe40b7fe640 1 -- 172.21.15.72:0/1183444834 learned_addr learned my addr 172.21.15.72:0/1183444834 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.974+0000 7fe41089e640 1 --2- 172.21.15.72:0/1183444834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c0c75f0 0x7fe40c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.974+0000 7fe40b7fe640 1 -- 172.21.15.72:0/1183444834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c0c75f0 msgr2=0x7fe40c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.974+0000 7fe40b7fe640 1 --2- 172.21.15.72:0/1183444834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c0c75f0 0x7fe40c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.974+0000 7fe40b7fe640 1 -- 172.21.15.72:0/1183444834 >> v1:172.21.15.113:6789/0 conn(0x7fe40c160810 legacy=0x7fe40c0c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.974+0000 7fe40b7fe640 1 -- 172.21.15.72:0/1183444834 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c0d0ae0 con 0x7fe40c0c7170 2023-12-12T09:52:04.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.974+0000 7fe41089e640 1 --2- 172.21.15.72:0/1183444834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c0c75f0 0x7fe40c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:04.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.975+0000 7fe40b7fe640 1 --2- 172.21.15.72:0/1183444834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c0cdab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009870 tx=0x7fe3f402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec57526b0089c8e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.975+0000 7fe40affd640 1 -- 172.21.15.72:0/1183444834 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40317c0 con 0x7fe40c0c7170 2023-12-12T09:52:04.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.975+0000 7fe40affd640 1 -- 172.21.15.72:0/1183444834 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f4031920 con 0x7fe40c0c7170 2023-12-12T09:52:04.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.975+0000 7fe40affd640 1 -- 172.21.15.72:0/1183444834 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4031cb0 con 0x7fe40c0c7170 2023-12-12T09:52:04.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.976+0000 7fe41109f640 1 -- 172.21.15.72:0/1183444834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 msgr2=0x7fe40c0cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.976+0000 7fe41109f640 1 --2- 172.21.15.72:0/1183444834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c0cdab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009870 tx=0x7fe3f402b860 comp rx=0 tx=0).stop 2023-12-12T09:52:04.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.976+0000 7fe41109f640 1 -- 172.21.15.72:0/1183444834 shutdown_connections 2023-12-12T09:52:04.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.976+0000 7fe41109f640 1 --2- 172.21.15.72:0/1183444834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c0c75f0 0x7fe40c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.976+0000 7fe41109f640 1 --2- 172.21.15.72:0/1183444834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c0cdab0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.976+0000 7fe41109f640 1 -- 172.21.15.72:0/1183444834 >> 172.21.15.72:0/1183444834 conn(0x7fe40c0d2f10 msgr2=0x7fe40c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:04.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.976+0000 7fe41109f640 1 -- 172.21.15.72:0/1183444834 shutdown_connections 2023-12-12T09:52:04.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.976+0000 7fe41109f640 1 -- 172.21.15.72:0/1183444834 wait complete. 2023-12-12T09:52:04.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.978+0000 7fe41109f640 1 Processor -- start 2023-12-12T09:52:04.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.978+0000 7fe41109f640 1 -- start start 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe41109f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe41109f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe40c0c75f0 0x7fe40c157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe41109f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c160810 0x7fe40c14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe41109f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c14b8e0 con 0x7fe40c0c7170 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe41109f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c158430 con 0x7fe40c0c75f0 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe41109f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c158710 con 0x7fe40c160810 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c1594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60826/0 (socket says 172.21.15.72:60826) 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 -- 172.21.15.72:0/891133209 learned_addr learned my addr 172.21.15.72:0/891133209 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:04.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40b7fe640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe40c0c75f0 0x7fe40c157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 -- 172.21.15.72:0/891133209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c160810 msgr2=0x7fe40c14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c160810 0x7fe40c14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 -- 172.21.15.72:0/891133209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe40c0c75f0 msgr2=0x7fe40c157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe40c0c75f0 0x7fe40c157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 -- 172.21.15.72:0/891133209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40094d0 con 0x7fe40c0c7170 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.979+0000 7fe40bfff640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c1594d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00c870 tx=0x7fe3fc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.980+0000 7fe408ff9640 1 -- 172.21.15.72:0/891133209 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc003b70 con 0x7fe40c0c7170 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.980+0000 7fe408ff9640 1 -- 172.21.15.72:0/891133209 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc003cf0 con 0x7fe40c0c7170 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.980+0000 7fe408ff9640 1 -- 172.21.15.72:0/891133209 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc00aa90 con 0x7fe40c0c7170 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.980+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c1589f0 con 0x7fe40c0c7170 2023-12-12T09:52:04.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.980+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c158fb0 con 0x7fe40c0c7170 2023-12-12T09:52:04.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.981+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c0673f0 con 0x7fe40c0c7170 2023-12-12T09:52:04.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.982+0000 7fe408ff9640 1 -- 172.21.15.72:0/891133209 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3fc00abf0 con 0x7fe40c0c7170 2023-12-12T09:52:04.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.982+0000 7fe408ff9640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3ec06ae40 0x7fe3ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:04.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.983+0000 7fe408ff9640 1 -- 172.21.15.72:0/891133209 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe3fc089220 con 0x7fe40c0c7170 2023-12-12T09:52:04.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.983+0000 7fe40b7fe640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3ec06ae40 0x7fe3ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:04.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.983+0000 7fe40b7fe640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3ec06ae40 0x7fe3ec06d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4036000 tx=0x7fe3f403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:04.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:04.987+0000 7fe408ff9640 1 -- 172.21.15.72:0/891133209 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3fc054590 con 0x7fe40c0c7170 2023-12-12T09:52:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.180+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe40c0d32f0 con 0x7fe40c0c7170 2023-12-12T09:52:05.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.180+0000 7fe408ff9640 1 -- 172.21.15.72:0/891133209 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe3fc003e60 con 0x7fe40c0c7170 2023-12-12T09:52:05.182 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:05.182 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2023-12-12T09:52:05.183 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:52:05.183 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 1, 2023-12-12T09:52:05.183 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": ".mgr" 2023-12-12T09:52:05.183 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:52:05.183 INFO:teuthology.orchestra.run.smithi072.stdout:] 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3ec06ae40 msgr2=0x7fe3ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3ec06ae40 0x7fe3ec06d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4036000 tx=0x7fe3f403d040 comp rx=0 tx=0).stop 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 msgr2=0x7fe40c1594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c1594d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00c870 tx=0x7fe3fc00cd30 comp rx=0 tx=0).stop 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 shutdown_connections 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3ec06ae40 0x7fe3ec06d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe40c160810 0x7fe40c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe40c0c75f0 0x7fe40c157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 --2- 172.21.15.72:0/891133209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe40c0c7170 0x7fe40c1594d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 >> 172.21.15.72:0/891133209 conn(0x7fe40c0d2f10 msgr2=0x7fe40c15f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:05.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 shutdown_connections 2023-12-12T09:52:05.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.183+0000 7fe41109f640 1 -- 172.21.15.72:0/891133209 wait complete. 2023-12-12T09:52:05.203 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-12T09:52:05.203 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-12T09:52:05.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.413+0000 7ff076e2e640 1 Processor -- start 2023-12-12T09:52:05.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.413+0000 7ff076e2e640 1 -- start start 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff076e2e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff0700ce1b0 0x7ff0700cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff076e2e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff0700cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff076e2e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07015c4b0 con 0x7ff0700b3770 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff076e2e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0700cd750 con 0x7ff0700ce1b0 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff076e2e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700b3b50 con 0x7ff0700ce630 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff075e2c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff0700b3770 0x7ff0700cb1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:34862/0 (socket says 172.21.15.72:34862) 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff075e2c640 1 -- 172.21.15.72:0/839227182 learned_addr learned my addr 172.21.15.72:0/839227182 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff07562b640 1 --2- 172.21.15.72:0/839227182 >> v2:172.21.15.72:3300/0 conn(0x7ff0700ce1b0 0x7ff0700cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff07662d640 1 --2- 172.21.15.72:0/839227182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff0700cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff074e2a640 1 -- 172.21.15.72:0/839227182 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957051584 0 0) 0x7ff06c001110 con 0x7ff0700b3770 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff074e2a640 1 -- 172.21.15.72:0/839227182 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff05c003520 con 0x7ff0700b3770 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.415+0000 7ff07562b640 1 -- 172.21.15.72:0/839227182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 msgr2=0x7ff0700cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:05.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.416+0000 7ff07562b640 1 --2- 172.21.15.72:0/839227182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff0700cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.416+0000 7ff07562b640 1 -- 172.21.15.72:0/839227182 >> v1:172.21.15.72:6789/0 conn(0x7ff0700b3770 legacy=0x7ff0700cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.416+0000 7ff07562b640 1 -- 172.21.15.72:0/839227182 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07015b930 con 0x7ff0700ce1b0 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.416+0000 7ff07662d640 1 --2- 172.21.15.72:0/839227182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff0700cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.417+0000 7ff07562b640 1 --2- 172.21.15.72:0/839227182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff0700cefb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff0600097a0 tx=0x7ff06002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc79f80aa532c6d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.417+0000 7ff074e2a640 1 -- 172.21.15.72:0/839227182 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06002d040 con 0x7ff0700ce1b0 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.417+0000 7ff074e2a640 1 -- 172.21.15.72:0/839227182 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff060031810 con 0x7ff0700ce1b0 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.417+0000 7ff074e2a640 1 -- 172.21.15.72:0/839227182 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060031ba0 con 0x7ff0700ce1b0 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.418+0000 7ff076e2e640 1 -- 172.21.15.72:0/839227182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 msgr2=0x7ff0700cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.418+0000 7ff076e2e640 1 --2- 172.21.15.72:0/839227182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff0700cefb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff0600097a0 tx=0x7ff06002b8b0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.418+0000 7ff076e2e640 1 -- 172.21.15.72:0/839227182 shutdown_connections 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.418+0000 7ff076e2e640 1 --2- 172.21.15.72:0/839227182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff0700cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.418+0000 7ff076e2e640 1 --2- 172.21.15.72:0/839227182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff0700cefb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.418+0000 7ff076e2e640 1 -- 172.21.15.72:0/839227182 >> 172.21.15.72:0/839227182 conn(0x7ff07006a880 msgr2=0x7ff07006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.418+0000 7ff076e2e640 1 -- 172.21.15.72:0/839227182 shutdown_connections 2023-12-12T09:52:05.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.418+0000 7ff076e2e640 1 -- 172.21.15.72:0/839227182 wait complete. 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.420+0000 7ff076e2e640 1 Processor -- start 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.420+0000 7ff076e2e640 1 -- start start 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff076e2e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0700b3770 0x7ff07009cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff076e2e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff07015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff076e2e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff070155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff076e2e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07009c690 con 0x7ff0700ce1b0 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff076e2e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07009bcf0 con 0x7ff0700b3770 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff076e2e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07009bfd0 con 0x7ff0700ce630 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff07562b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff07015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff07562b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff07015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40078/0 (socket says 172.21.15.72:40078) 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff07562b640 1 -- 172.21.15.72:0/1490110059 learned_addr learned my addr 172.21.15.72:0/1490110059 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:05.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff075e2c640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0700b3770 0x7ff07009cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff07662d640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff070155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff07562b640 1 -- 172.21.15.72:0/1490110059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 msgr2=0x7ff070155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff07562b640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff070155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff07562b640 1 -- 172.21.15.72:0/1490110059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0700b3770 msgr2=0x7ff07009cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.421+0000 7ff07562b640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0700b3770 0x7ff07009cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.422+0000 7ff07562b640 1 -- 172.21.15.72:0/1490110059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff060009400 con 0x7ff0700ce1b0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.422+0000 7ff07562b640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff07015a390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff0600098c0 tx=0x7ff0600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.422+0000 7ff0667fc640 1 -- 172.21.15.72:0/1490110059 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060006e10 con 0x7ff0700ce1b0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.422+0000 7ff0667fc640 1 -- 172.21.15.72:0/1490110059 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06003ad60 con 0x7ff0700ce1b0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.422+0000 7ff0667fc640 1 -- 172.21.15.72:0/1490110059 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060044620 con 0x7ff0700ce1b0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.422+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07009c2b0 con 0x7ff0700ce1b0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.422+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07015a8d0 con 0x7ff0700ce1b0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.423+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07015aac0 con 0x7ff0700ce1b0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.424+0000 7ff0667fc640 1 -- 172.21.15.72:0/1490110059 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff06004d440 con 0x7ff0700ce1b0 2023-12-12T09:52:05.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.425+0000 7ff0667fc640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff05406ae40 0x7ff05406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:05.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.425+0000 7ff0667fc640 1 -- 172.21.15.72:0/1490110059 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff0600ab6c0 con 0x7ff0700ce1b0 2023-12-12T09:52:05.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.425+0000 7ff075e2c640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff05406ae40 0x7ff05406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:05.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.426+0000 7ff075e2c640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff05406ae40 0x7ff05406d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff06c007fd0 tx=0x7ff06c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:05.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.430+0000 7ff0667fc640 1 -- 172.21.15.72:0/1490110059 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff060076a30 con 0x7ff0700ce1b0 2023-12-12T09:52:05.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.623+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff070074450 con 0x7ff0700ce1b0 2023-12-12T09:52:05.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.955+0000 7ff0667fc640 1 -- 172.21.15.72:0/1490110059 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff06007a050 con 0x7ff0700ce1b0 2023-12-12T09:52:05.959 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' created 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.958+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff05406ae40 msgr2=0x7ff05406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.958+0000 7ff076e2e640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff05406ae40 0x7ff05406d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff06c007fd0 tx=0x7ff06c009040 comp rx=0 tx=0).stop 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 msgr2=0x7ff07015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff07015a390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff0600098c0 tx=0x7ff0600046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 shutdown_connections 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff05406ae40 0x7ff05406d300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0700ce630 0x7ff070155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0700ce1b0 0x7ff07015a390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 --2- 172.21.15.72:0/1490110059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0700b3770 0x7ff07009cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 >> 172.21.15.72:0/1490110059 conn(0x7ff07006a880 msgr2=0x7ff07015e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:05.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 shutdown_connections 2023-12-12T09:52:05.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:05.959+0000 7ff076e2e640 1 -- 172.21.15.72:0/1490110059 wait complete. 2023-12-12T09:52:05.981 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-12-12T09:52:06.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.198+0000 7fc231533640 1 Processor -- start 2023-12-12T09:52:06.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.199+0000 7fc231533640 1 -- start start 2023-12-12T09:52:06.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc231533640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15d240 0x7fc22c070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:06.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc231533640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc22c0ac860 0x7fc22c070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:06.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc231533640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c071800 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc231533640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c15d620 con 0x7fc22c0ac860 2023-12-12T09:52:06.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc231533640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c070f60 con 0x7fc22c15d240 2023-12-12T09:52:06.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc22affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15d240 0x7fc22c070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:06.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc22affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15d240 0x7fc22c070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:55024/0 (socket says 172.21.15.72:55024) 2023-12-12T09:52:06.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc22affd640 1 -- 172.21.15.72:0/3429782805 learned_addr learned my addr 172.21.15.72:0/3429782805 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:06.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc22affd640 1 -- 172.21.15.72:0/3429782805 >> v2:172.21.15.113:3300/0 conn(0x7fc22c0ac860 msgr2=0x7fc22c070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:06.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.201+0000 7fc22affd640 1 --2- 172.21.15.72:0/3429782805 >> v2:172.21.15.113:3300/0 conn(0x7fc22c0ac860 0x7fc22c070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.202+0000 7fc22affd640 1 -- 172.21.15.72:0/3429782805 >> v1:172.21.15.113:6789/0 conn(0x7fc22c0cf1a0 legacy=0x7fc22c160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:06.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.202+0000 7fc22affd640 1 -- 172.21.15.72:0/3429782805 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c071240 con 0x7fc22c15d240 2023-12-12T09:52:06.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.203+0000 7fc22affd640 1 --2- 172.21.15.72:0/3429782805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15d240 0x7fc22c070410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009870 tx=0x7fc21c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82554a7751c4bca1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:06.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.203+0000 7fc22a7fc640 1 -- 172.21.15.72:0/3429782805 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0317c0 con 0x7fc22c15d240 2023-12-12T09:52:06.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.203+0000 7fc22a7fc640 1 -- 172.21.15.72:0/3429782805 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c031920 con 0x7fc22c15d240 2023-12-12T09:52:06.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.203+0000 7fc22a7fc640 1 -- 172.21.15.72:0/3429782805 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c031cb0 con 0x7fc22c15d240 2023-12-12T09:52:06.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.204+0000 7fc231533640 1 -- 172.21.15.72:0/3429782805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15d240 msgr2=0x7fc22c070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:06.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.204+0000 7fc231533640 1 --2- 172.21.15.72:0/3429782805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15d240 0x7fc22c070410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009870 tx=0x7fc21c02b860 comp rx=0 tx=0).stop 2023-12-12T09:52:06.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.204+0000 7fc231533640 1 -- 172.21.15.72:0/3429782805 shutdown_connections 2023-12-12T09:52:06.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.204+0000 7fc231533640 1 --2- 172.21.15.72:0/3429782805 >> v2:172.21.15.113:3300/0 conn(0x7fc22c0ac860 0x7fc22c070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.204+0000 7fc231533640 1 --2- 172.21.15.72:0/3429782805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c15d240 0x7fc22c070410 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.204+0000 7fc231533640 1 -- 172.21.15.72:0/3429782805 >> 172.21.15.72:0/3429782805 conn(0x7fc22c06a880 msgr2=0x7fc22c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:06.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.205+0000 7fc231533640 1 -- 172.21.15.72:0/3429782805 shutdown_connections 2023-12-12T09:52:06.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.205+0000 7fc231533640 1 -- 172.21.15.72:0/3429782805 wait complete. 2023-12-12T09:52:06.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.207+0000 7fc231533640 1 Processor -- start 2023-12-12T09:52:06.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.207+0000 7fc231533640 1 -- start start 2023-12-12T09:52:06.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.208+0000 7fc231533640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c0ac860 0x7fc22c1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:06.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.208+0000 7fc231533640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc22c0cf1a0 0x7fc22c0ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:06.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.208+0000 7fc231533640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c15d240 0x7fc22c0af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:06.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.208+0000 7fc231533640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0adcd0 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.208+0000 7fc231533640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0ad050 con 0x7fc22c15d240 2023-12-12T09:52:06.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.208+0000 7fc231533640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0ad330 con 0x7fc22c0ac860 2023-12-12T09:52:06.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.209+0000 7fc223fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc22c0cf1a0 0x7fc22c0ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:06.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.209+0000 7fc223fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc22c0cf1a0 0x7fc22c0ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40096/0 (socket says 172.21.15.72:40096) 2023-12-12T09:52:06.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.209+0000 7fc223fff640 1 -- 172.21.15.72:0/352603623 learned_addr learned my addr 172.21.15.72:0/352603623 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.209+0000 7fc223fff640 1 -- 172.21.15.72:0/352603623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c0ac860 msgr2=0x7fc22c1445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.209+0000 7fc223fff640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c0ac860 0x7fc22c1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.209+0000 7fc223fff640 1 -- 172.21.15.72:0/352603623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c15d240 msgr2=0x7fc22c0af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.209+0000 7fc223fff640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c15d240 0x7fc22c0af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.209+0000 7fc223fff640 1 -- 172.21.15.72:0/352603623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0094d0 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.210+0000 7fc223fff640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc22c0cf1a0 0x7fc22c0ae0f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc21400b410 tx=0x7fc21400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.210+0000 7fc2237fe640 1 -- 172.21.15.72:0/352603623 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214011830 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.210+0000 7fc2237fe640 1 -- 172.21.15.72:0/352603623 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214011990 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.210+0000 7fc2237fe640 1 -- 172.21.15.72:0/352603623 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214011d40 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.210+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c0ad610 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.210+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c0acd20 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.211+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c0ade10 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.212+0000 7fc2237fe640 1 -- 172.21.15.72:0/352603623 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc21401a400 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.213+0000 7fc2237fe640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc20c06ae40 0x7fc20c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:06.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.214+0000 7fc2237fe640 1 -- 172.21.15.72:0/352603623 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc214089550 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.214+0000 7fc22affd640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc20c06ae40 0x7fc20c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:06.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.214+0000 7fc22affd640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc20c06ae40 0x7fc20c06d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc21c036000 tx=0x7fc21c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:06.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.218+0000 7fc2237fe640 1 -- 172.21.15.72:0/352603623 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc22c0ade10 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.409+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc22c0ade10 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.962+0000 7fc2237fe640 1 -- 172.21.15.72:0/352603623 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc22c0ade10 con 0x7fc22c0cf1a0 2023-12-12T09:52:06.965 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' created 2023-12-12T09:52:06.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.967+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc20c06ae40 msgr2=0x7fc20c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:06.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.967+0000 7fc231533640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc20c06ae40 0x7fc20c06d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc21c036000 tx=0x7fc21c030000 comp rx=0 tx=0).stop 2023-12-12T09:52:06.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.967+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc22c0cf1a0 msgr2=0x7fc22c0ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:06.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.967+0000 7fc231533640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc22c0cf1a0 0x7fc22c0ae0f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc21400b410 tx=0x7fc21400b8d0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.968+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 shutdown_connections 2023-12-12T09:52:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.968+0000 7fc231533640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc20c06ae40 0x7fc20c06d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.968+0000 7fc231533640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c15d240 0x7fc22c0af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.968+0000 7fc231533640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc22c0cf1a0 0x7fc22c0ae0f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.968+0000 7fc231533640 1 --2- 172.21.15.72:0/352603623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc22c0ac860 0x7fc22c1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.968+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 >> 172.21.15.72:0/352603623 conn(0x7fc22c06a880 msgr2=0x7fc22c15f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.968+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 shutdown_connections 2023-12-12T09:52:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:06.968+0000 7fc231533640 1 -- 172.21.15.72:0/352603623 wait complete. 2023-12-12T09:52:07.010 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-12T09:52:07.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.192+0000 7fe3fee9b640 1 Processor -- start 2023-12-12T09:52:07.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.192+0000 7fe3fee9b640 1 -- start start 2023-12-12T09:52:07.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.193+0000 7fe3fee9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3f815a1e0 0x7fe3f806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:07.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.193+0000 7fe3fee9b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe3f80ae300 0x7fe3f806a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:07.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.193+0000 7fe3fee9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f8070370 0x7fe3f806a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:07.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.193+0000 7fe3fee9b640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80cdaa0 con 0x7fe3f80ae300 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.193+0000 7fe3fee9b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80cdd80 con 0x7fe3f815a1e0 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.193+0000 7fe3fee9b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80ce060 con 0x7fe3f8070370 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3effff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe3f80ae300 0x7fe3f806a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3effff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe3f80ae300 0x7fe3f806a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40098/0 (socket says 172.21.15.72:40098) 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3effff640 1 -- 172.21.15.72:0/1865328545 learned_addr learned my addr 172.21.15.72:0/1865328545 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3fcc37640 1 --2- 172.21.15.72:0/1865328545 >> v2:172.21.15.113:3300/0 conn(0x7fe3f815a1e0 0x7fe3f806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3fd438640 1 --2- 172.21.15.72:0/1865328545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f8070370 0x7fe3f806a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3effff640 1 -- 172.21.15.72:0/1865328545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f8070370 msgr2=0x7fe3f806a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3effff640 1 --2- 172.21.15.72:0/1865328545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f8070370 0x7fe3f806a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3effff640 1 -- 172.21.15.72:0/1865328545 >> v2:172.21.15.113:3300/0 conn(0x7fe3f815a1e0 msgr2=0x7fe3f806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3effff640 1 --2- 172.21.15.72:0/1865328545 >> v2:172.21.15.113:3300/0 conn(0x7fe3f815a1e0 0x7fe3f806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:07.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.194+0000 7fe3effff640 1 -- 172.21.15.72:0/1865328545 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f80ce340 con 0x7fe3f80ae300 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.195+0000 7fe3effff640 1 --2- 172.21.15.72:0/1865328545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f80ae300 0x7fe3f806a460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80b1800 tx=0x7fe3e802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbe574c7fb4d58e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3ef7fe640 1 -- 172.21.15.72:0/1865328545 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80317e0 con 0x7fe3f80ae300 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3ef7fe640 1 -- 172.21.15.72:0/1865328545 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e8031940 con 0x7fe3f80ae300 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3ef7fe640 1 -- 172.21.15.72:0/1865328545 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8031cd0 con 0x7fe3f80ae300 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1865328545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f80ae300 msgr2=0x7fe3f806a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1865328545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f80ae300 0x7fe3f806a460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80b1800 tx=0x7fe3e802b880 comp rx=0 tx=0).stop 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1865328545 shutdown_connections 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1865328545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f8070370 0x7fe3f806a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1865328545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f80ae300 0x7fe3f806a460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1865328545 >> v2:172.21.15.113:3300/0 conn(0x7fe3f815a1e0 0x7fe3f806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:07.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1865328545 >> 172.21.15.72:0/1865328545 conn(0x7fe3f80cee70 msgr2=0x7fe3f80cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:07.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1865328545 shutdown_connections 2023-12-12T09:52:07.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.196+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1865328545 wait complete. 2023-12-12T09:52:07.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.198+0000 7fe3fee9b640 1 Processor -- start 2023-12-12T09:52:07.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.198+0000 7fe3fee9b640 1 -- start start 2023-12-12T09:52:07.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fee9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f8070370 0x7fe3f815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:07.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fee9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f80ae300 0x7fe3f809b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:07.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fee9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f815a1e0 0x7fe3f8156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:07.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fee9b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f815b3e0 con 0x7fe3f8070370 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fee9b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80c7c40 con 0x7fe3f80ae300 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fee9b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80c7f20 con 0x7fe3f815a1e0 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fcc37640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f8070370 0x7fe3f815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fcc37640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f8070370 0x7fe3f815aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40112/0 (socket says 172.21.15.72:40112) 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.199+0000 7fe3fcc37640 1 -- 172.21.15.72:0/1265099271 learned_addr learned my addr 172.21.15.72:0/1265099271 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3fd438640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f815a1e0 0x7fe3f8156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3effff640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f80ae300 0x7fe3f809b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3fcc37640 1 -- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f815a1e0 msgr2=0x7fe3f8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3fcc37640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f815a1e0 0x7fe3f8156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3fcc37640 1 -- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f80ae300 msgr2=0x7fe3f809b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3fcc37640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f80ae300 0x7fe3f809b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3fcc37640 1 -- 172.21.15.72:0/1265099271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80094d0 con 0x7fe3f8070370 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3effff640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f80ae300 0x7fe3f809b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:07.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3fcc37640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f8070370 0x7fe3f815aea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400d5f0 tx=0x7fe3f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:07.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3ed7fa640 1 -- 172.21.15.72:0/1265099271 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40125f0 con 0x7fe3f8070370 2023-12-12T09:52:07.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3ed7fa640 1 -- 172.21.15.72:0/1265099271 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f4012750 con 0x7fe3f8070370 2023-12-12T09:52:07.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3ed7fa640 1 -- 172.21.15.72:0/1265099271 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4012b00 con 0x7fe3f8070370 2023-12-12T09:52:07.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.200+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f80c5e90 con 0x7fe3f8070370 2023-12-12T09:52:07.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.201+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f80c8200 con 0x7fe3f8070370 2023-12-12T09:52:07.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.202+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f80ca210 con 0x7fe3f8070370 2023-12-12T09:52:07.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.203+0000 7fe3ed7fa640 1 -- 172.21.15.72:0/1265099271 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3f4012c60 con 0x7fe3f8070370 2023-12-12T09:52:07.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.204+0000 7fe3ed7fa640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3e406ae40 0x7fe3e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:07.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.204+0000 7fe3ed7fa640 1 -- 172.21.15.72:0/1265099271 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe3f40899f0 con 0x7fe3f8070370 2023-12-12T09:52:07.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.204+0000 7fe3effff640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3e406ae40 0x7fe3e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:07.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.205+0000 7fe3effff640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3e406ae40 0x7fe3e406d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8036000 tx=0x7fe3e803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:07.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.209+0000 7fe3ed7fa640 1 -- 172.21.15.72:0/1265099271 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3f4054b90 con 0x7fe3f8070370 2023-12-12T09:52:07.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:07.434+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe3f8074450 con 0x7fe3f8070370 2023-12-12T09:52:08.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.005+0000 7fe3ed7fa640 1 -- 172.21.15.72:0/1265099271 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe3f40581b0 con 0x7fe3f8070370 2023-12-12T09:52:08.010 INFO:teuthology.orchestra.run.smithi072.stderr:new fs with metadata pool 2 and data pool 3 2023-12-12T09:52:08.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.022+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3e406ae40 msgr2=0x7fe3e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:08.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.022+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3e406ae40 0x7fe3e406d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8036000 tx=0x7fe3e803d040 comp rx=0 tx=0).stop 2023-12-12T09:52:08.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.022+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f8070370 msgr2=0x7fe3f815aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:08.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.022+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f8070370 0x7fe3f815aea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400d5f0 tx=0x7fe3f400dab0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.023+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 shutdown_connections 2023-12-12T09:52:08.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.023+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3e406ae40 0x7fe3e406d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.023+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f815a1e0 0x7fe3f8156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.023+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f80ae300 0x7fe3f809b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.023+0000 7fe3fee9b640 1 --2- 172.21.15.72:0/1265099271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f8070370 0x7fe3f815aea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.023+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 >> 172.21.15.72:0/1265099271 conn(0x7fe3f80cee70 msgr2=0x7fe3f80afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:08.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.024+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 shutdown_connections 2023-12-12T09:52:08.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.024+0000 7fe3fee9b640 1 -- 172.21.15.72:0/1265099271 wait complete. 2023-12-12T09:52:08.057 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-12-12T09:52:08.057 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-12-12T09:52:08.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.243+0000 7f23d5388640 1 Processor -- start 2023-12-12T09:52:08.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.243+0000 7f23d5388640 1 -- start start 2023-12-12T09:52:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.245+0000 7f23d5388640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23d0161510 0x7f23d00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.245+0000 7f23d5388640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d00cf6c0 0x7f23d01651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.245+0000 7f23d5388640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00cbef0 con 0x7f23d00cd6a0 2023-12-12T09:52:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.245+0000 7f23d5388640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d00b2e50 con 0x7f23d0161510 2023-12-12T09:52:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.245+0000 7f23d5388640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00ce350 con 0x7f23d00cf6c0 2023-12-12T09:52:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.245+0000 7f23c77fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f23d00cd6a0 0x7f23d00cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:34878/0 (socket says 172.21.15.72:34878) 2023-12-12T09:52:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.245+0000 7f23c77fe640 1 -- 172.21.15.72:0/570187811 learned_addr learned my addr 172.21.15.72:0/570187811 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23d4b87640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d00cf6c0 0x7f23d01651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cffff640 1 --2- 172.21.15.72:0/570187811 >> v2:172.21.15.113:3300/0 conn(0x7f23d0161510 0x7f23d00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5188223 0 0) 0x7f23d00cbef0 con 0x7f23d00cd6a0 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23bc003520 con 0x7f23d00cd6a0 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3418152228 0 0) 0x7f23bc003520 con 0x7f23d00cd6a0 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d00cf6c0 msgr2=0x7f23d01651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d00cf6c0 0x7f23d01651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 >> v2:172.21.15.113:3300/0 conn(0x7f23d0161510 msgr2=0x7f23d00cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 --2- 172.21.15.72:0/570187811 >> v2:172.21.15.113:3300/0 conn(0x7f23d0161510 0x7f23d00cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00cb5c0 con 0x7f23d00cd6a0 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f23b8001670 con 0x7f23d00cd6a0 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 >> v1:172.21.15.72:6789/0 conn(0x7f23d00cd6a0 legacy=0x7f23d00cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc003970 0x7f23bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc0047c0 0x7f23bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d00cf6c0 0x7f23bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00b2e50 con 0x7f23bc0047c0 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00ce350 con 0x7f23bc003970 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.246+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc00b980 con 0x7f23d00cf6c0 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.247+0000 7f23c77fe640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc0047c0 0x7f23bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.247+0000 7f23cffff640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc003970 0x7f23bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.247+0000 7f23d4b87640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d00cf6c0 0x7f23bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.247+0000 7f23c77fe640 1 -- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d00cf6c0 msgr2=0x7f23bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:08.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.247+0000 7f23c77fe640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d00cf6c0 0x7f23bc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.247+0000 7f23c77fe640 1 -- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc003970 msgr2=0x7f23bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:08.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.247+0000 7f23c77fe640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc003970 0x7f23bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.247+0000 7f23c77fe640 1 -- 172.21.15.72:0/570187811 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc003520 con 0x7f23bc0047c0 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.248+0000 7f23c77fe640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc0047c0 0x7f23bc004c40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23b800c160 tx=0x7f23bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e821a90854e7a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.248+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8005ad0 con 0x7f23bc0047c0 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.248+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b8005c30 con 0x7f23bc0047c0 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.248+0000 7f23cf7fe640 1 -- 172.21.15.72:0/570187811 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b802c810 con 0x7f23bc0047c0 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.249+0000 7f23d5388640 1 -- 172.21.15.72:0/570187811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc0047c0 msgr2=0x7f23bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.249+0000 7f23d5388640 1 --2- 172.21.15.72:0/570187811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc0047c0 0x7f23bc004c40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23b800c160 tx=0x7f23bc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.249+0000 7f23d4b87640 1 -- 172.21.15.72:0/570187811 reap_dead start 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.249+0000 7f23d5388640 1 -- 172.21.15.72:0/570187811 shutdown_connections 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.249+0000 7f23d5388640 1 -- 172.21.15.72:0/570187811 >> 172.21.15.72:0/570187811 conn(0x7f23d006a880 msgr2=0x7f23d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.249+0000 7f23d5388640 1 -- 172.21.15.72:0/570187811 shutdown_connections 2023-12-12T09:52:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.249+0000 7f23d5388640 1 -- 172.21.15.72:0/570187811 wait complete. 2023-12-12T09:52:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.251+0000 7f23d5388640 1 Processor -- start 2023-12-12T09:52:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.251+0000 7f23d5388640 1 -- start start 2023-12-12T09:52:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23d5388640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc0047c0 0x7f23d014bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23d5388640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23d014c400 0x7f23d014ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23d5388640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d014d960 0x7f23d014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23d5388640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00afe90 con 0x7f23d014c400 2023-12-12T09:52:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23d5388640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d014f390 con 0x7f23bc0047c0 2023-12-12T09:52:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23d5388640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d014c8c0 con 0x7f23d014d960 2023-12-12T09:52:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23c77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23d014c400 0x7f23d014ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23c77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23d014c400 0x7f23d014ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40138/0 (socket says 172.21.15.72:40138) 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.252+0000 7f23c77fe640 1 -- 172.21.15.72:0/2284624544 learned_addr learned my addr 172.21.15.72:0/2284624544 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23d4b87640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d014d960 0x7f23d014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23cffff640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc0047c0 0x7f23d014bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23c77fe640 1 -- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d014d960 msgr2=0x7f23d014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23c77fe640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d014d960 0x7f23d014de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23c77fe640 1 -- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc0047c0 msgr2=0x7f23d014bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23c77fe640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc0047c0 0x7f23d014bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23c77fe640 1 -- 172.21.15.72:0/2284624544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d014cba0 con 0x7f23d014c400 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23cffff640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc0047c0 0x7f23d014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23c77fe640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23d014c400 0x7f23d014ee50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f23b8000fb0 tx=0x7f23b8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23cd7fa640 1 -- 172.21.15.72:0/2284624544 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8039070 con 0x7f23d014c400 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23cd7fa640 1 -- 172.21.15.72:0/2284624544 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b802cc20 con 0x7f23d014c400 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23cd7fa640 1 -- 172.21.15.72:0/2284624544 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8007070 con 0x7f23d014c400 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.253+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d014ce80 con 0x7f23d014c400 2023-12-12T09:52:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.254+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d014d440 con 0x7f23d014c400 2023-12-12T09:52:08.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.255+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d014d5d0 con 0x7f23d014c400 2023-12-12T09:52:08.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.256+0000 7f23cd7fa640 1 -- 172.21.15.72:0/2284624544 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f23b8032070 con 0x7f23d014c400 2023-12-12T09:52:08.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.256+0000 7f23cd7fa640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f23b006ae40 0x7f23b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:08.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.257+0000 7f23cd7fa640 1 -- 172.21.15.72:0/2284624544 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23b803c040 con 0x7f23d014c400 2023-12-12T09:52:08.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.257+0000 7f23cffff640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f23b006ae40 0x7f23b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:08.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.257+0000 7f23cffff640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f23b006ae40 0x7f23b006d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009a00 tx=0x7f23c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:08.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.261+0000 7f23cd7fa640 1 -- 172.21.15.72:0/2284624544 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23b80758c0 con 0x7f23d014c400 2023-12-12T09:52:08.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:08.457+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f23d00cb280 con 0x7f23d014c400 2023-12-12T09:52:09.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.008+0000 7f23cd7fa640 1 -- 172.21.15.72:0/2284624544 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f23b8078ee0 con 0x7f23d014c400 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.011+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f23b006ae40 msgr2=0x7f23b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.011+0000 7f23d5388640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f23b006ae40 0x7f23b006d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009a00 tx=0x7f23c0008040 comp rx=0 tx=0).stop 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.011+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23d014c400 msgr2=0x7f23d014ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.011+0000 7f23d5388640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23d014c400 0x7f23d014ee50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f23b8000fb0 tx=0x7f23b8000fe0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.012+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 shutdown_connections 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.012+0000 7f23d5388640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f23b006ae40 0x7f23b006d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.012+0000 7f23d5388640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23d014d960 0x7f23d014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.012+0000 7f23d5388640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23d014c400 0x7f23d014ee50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.012+0000 7f23d5388640 1 --2- 172.21.15.72:0/2284624544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23bc0047c0 0x7f23d014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.012+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 >> 172.21.15.72:0/2284624544 conn(0x7f23d006a880 msgr2=0x7f23d00aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.012+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 shutdown_connections 2023-12-12T09:52:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.012+0000 7f23d5388640 1 -- 172.21.15.72:0/2284624544 wait complete. 2023-12-12T09:52:09.033 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-12-12T09:52:09.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.245+0000 7fdbaec91640 1 Processor -- start 2023-12-12T09:52:09.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.245+0000 7fdbaec91640 1 -- start start 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.246+0000 7fdbaec91640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdba815ba90 0x7fdba806d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.246+0000 7fdbaec91640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdba80cc2f0 0x7fdba8161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.246+0000 7fdbaec91640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba806d950 con 0x7fdba80b2aa0 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.246+0000 7fdbaec91640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba80b3fe0 con 0x7fdba80cc2f0 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.246+0000 7fdbaec91640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba80cc720 con 0x7fdba815ba90 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.247+0000 7fdbae490640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdba80cc2f0 0x7fdba8161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.247+0000 7fdbae490640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdba80cc2f0 0x7fdba8161a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40150/0 (socket says 172.21.15.72:40150) 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.247+0000 7fdbae490640 1 -- 172.21.15.72:0/285741874 learned_addr learned my addr 172.21.15.72:0/285741874 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.247+0000 7fdbadc8f640 1 --2- 172.21.15.72:0/285741874 >> v2:172.21.15.113:3300/0 conn(0x7fdba815ba90 0x7fdba806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.247+0000 7fdbae490640 1 -- 172.21.15.72:0/285741874 >> v2:172.21.15.113:3300/0 conn(0x7fdba815ba90 msgr2=0x7fdba806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.247+0000 7fdbae490640 1 --2- 172.21.15.72:0/285741874 >> v2:172.21.15.113:3300/0 conn(0x7fdba815ba90 0x7fdba806d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.247+0000 7fdbae490640 1 -- 172.21.15.72:0/285741874 >> v1:172.21.15.113:6789/0 conn(0x7fdba80b2aa0 legacy=0x7fdba80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.247+0000 7fdbae490640 1 -- 172.21.15.72:0/285741874 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba806c0b0 con 0x7fdba80cc2f0 2023-12-12T09:52:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.248+0000 7fdbae490640 1 --2- 172.21.15.72:0/285741874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80cc2f0 0x7fdba8161a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdba806bba0 tx=0x7fdba002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179addc99242929c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbacc8d640 1 -- 172.21.15.72:0/285741874 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba002c9f0 con 0x7fdba80cc2f0 2023-12-12T09:52:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbacc8d640 1 -- 172.21.15.72:0/285741874 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba002cb50 con 0x7fdba80cc2f0 2023-12-12T09:52:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbacc8d640 1 -- 172.21.15.72:0/285741874 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00343f0 con 0x7fdba80cc2f0 2023-12-12T09:52:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbaec91640 1 -- 172.21.15.72:0/285741874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80cc2f0 msgr2=0x7fdba8161a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbaec91640 1 --2- 172.21.15.72:0/285741874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80cc2f0 0x7fdba8161a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdba806bba0 tx=0x7fdba002be50 comp rx=0 tx=0).stop 2023-12-12T09:52:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbaec91640 1 -- 172.21.15.72:0/285741874 shutdown_connections 2023-12-12T09:52:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbaec91640 1 --2- 172.21.15.72:0/285741874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80cc2f0 0x7fdba8161a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbaec91640 1 --2- 172.21.15.72:0/285741874 >> v2:172.21.15.113:3300/0 conn(0x7fdba815ba90 0x7fdba806d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.249+0000 7fdbaec91640 1 -- 172.21.15.72:0/285741874 >> 172.21.15.72:0/285741874 conn(0x7fdba80b0940 msgr2=0x7fdba80b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.250+0000 7fdbaec91640 1 -- 172.21.15.72:0/285741874 shutdown_connections 2023-12-12T09:52:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.250+0000 7fdbaec91640 1 -- 172.21.15.72:0/285741874 wait complete. 2023-12-12T09:52:09.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.251+0000 7fdbaec91640 1 Processor -- start 2023-12-12T09:52:09.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.251+0000 7fdbaec91640 1 -- start start 2023-12-12T09:52:09.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.252+0000 7fdbaec91640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80b2aa0 0x7fdba81fb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:09.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.252+0000 7fdbaec91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba80cc2f0 0x7fdba8158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:09.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.252+0000 7fdbaec91640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba815ba90 0x7fdba80ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.252+0000 7fdbaec91640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba8158da0 con 0x7fdba80b2aa0 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.252+0000 7fdbaec91640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80cca90 con 0x7fdba80cc2f0 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.252+0000 7fdbaec91640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba809bdb0 con 0x7fdba815ba90 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80b2aa0 0x7fdba81fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80b2aa0 0x7fdba81fb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40162/0 (socket says 172.21.15.72:40162) 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 -- 172.21.15.72:0/4226107573 learned_addr learned my addr 172.21.15.72:0/4226107573 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbae490640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba815ba90 0x7fdba80ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbad48e640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba80cc2f0 0x7fdba8158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 -- 172.21.15.72:0/4226107573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba815ba90 msgr2=0x7fdba80ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba815ba90 0x7fdba80ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 -- 172.21.15.72:0/4226107573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba80cc2f0 msgr2=0x7fdba8158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:09.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba80cc2f0 0x7fdba8158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:09.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 -- 172.21.15.72:0/4226107573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba0009b30 con 0x7fdba80b2aa0 2023-12-12T09:52:09.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdbadc8f640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80b2aa0 0x7fdba81fb430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdba400c7f0 tx=0x7fdba400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:09.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.253+0000 7fdb9e7fc640 1 -- 172.21.15.72:0/4226107573 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4004220 con 0x7fdba80b2aa0 2023-12-12T09:52:09.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.254+0000 7fdb9e7fc640 1 -- 172.21.15.72:0/4226107573 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba4004380 con 0x7fdba80b2aa0 2023-12-12T09:52:09.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.254+0000 7fdb9e7fc640 1 -- 172.21.15.72:0/4226107573 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4007840 con 0x7fdba80b2aa0 2023-12-12T09:52:09.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.254+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba809c090 con 0x7fdba80b2aa0 2023-12-12T09:52:09.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.254+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba809c650 con 0x7fdba80b2aa0 2023-12-12T09:52:09.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.255+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba8067910 con 0x7fdba80b2aa0 2023-12-12T09:52:09.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.256+0000 7fdb9e7fc640 1 -- 172.21.15.72:0/4226107573 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdba4007bb0 con 0x7fdba80b2aa0 2023-12-12T09:52:09.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.257+0000 7fdb9e7fc640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb8c06ae70 0x7fdb8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:09.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.257+0000 7fdb9e7fc640 1 -- 172.21.15.72:0/4226107573 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fdba4089500 con 0x7fdba80b2aa0 2023-12-12T09:52:09.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.258+0000 7fdbad48e640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb8c06ae70 0x7fdb8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:09.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.258+0000 7fdbad48e640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb8c06ae70 0x7fdb8c06d330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdba81fa980 tx=0x7fdb98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:09.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.262+0000 7fdb9e7fc640 1 -- 172.21.15.72:0/4226107573 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdba4054560 con 0x7fdba80b2aa0 2023-12-12T09:52:09.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:09.452+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdba80cb680 con 0x7fdba80b2aa0 2023-12-12T09:52:11.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.032+0000 7fdb9e7fc640 1 -- 172.21.15.72:0/4226107573 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fdba4007e60 con 0x7fdba80b2aa0 2023-12-12T09:52:11.035 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data_ec' created 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb8c06ae70 msgr2=0x7fdb8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb8c06ae70 0x7fdb8c06d330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdba81fa980 tx=0x7fdb98009040 comp rx=0 tx=0).stop 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80b2aa0 msgr2=0x7fdba81fb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80b2aa0 0x7fdba81fb430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdba400c7f0 tx=0x7fdba400ccb0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 shutdown_connections 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb8c06ae70 0x7fdb8c06d330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba815ba90 0x7fdba80ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba80cc2f0 0x7fdba8158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 --2- 172.21.15.72:0/4226107573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdba80b2aa0 0x7fdba81fb430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.036+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 >> 172.21.15.72:0/4226107573 conn(0x7fdba80b0940 msgr2=0x7fdba806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:11.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.037+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 shutdown_connections 2023-12-12T09:52:11.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.037+0000 7fdbaec91640 1 -- 172.21.15.72:0/4226107573 wait complete. 2023-12-12T09:52:11.066 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-12-12T09:52:11.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.255+0000 7f565e961640 1 Processor -- start 2023-12-12T09:52:11.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.255+0000 7f565e961640 1 -- start start 2023-12-12T09:52:11.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f565e961640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f565815db60 0x7f565815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:11.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f565e961640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56580cc980 0x7f56580ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:11.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f565e961640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5658070fc0 con 0x7f5658159ed0 2023-12-12T09:52:11.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f565e961640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565815e530 con 0x7f565815db60 2023-12-12T09:52:11.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f565e961640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56580712a0 con 0x7f56580cc980 2023-12-12T09:52:11.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f56577fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f565815db60 0x7f565815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:11.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f56577fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f565815db60 0x7f565815dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40172/0 (socket says 172.21.15.72:40172) 2023-12-12T09:52:11.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f56577fe640 1 -- 172.21.15.72:0/1769309802 learned_addr learned my addr 172.21.15.72:0/1769309802 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:11.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f565cefe640 1 --2- 172.21.15.72:0/1769309802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56580cc980 0x7f56580ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:11.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f56577fe640 1 -- 172.21.15.72:0/1769309802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56580cc980 msgr2=0x7f56580ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:11.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f56577fe640 1 --2- 172.21.15.72:0/1769309802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56580cc980 0x7f56580ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.257+0000 7f56577fe640 1 -- 172.21.15.72:0/1769309802 >> v1:172.21.15.72:6789/0 conn(0x7f5658159ed0 legacy=0x7f565806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:11.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.258+0000 7f56577fe640 1 -- 172.21.15.72:0/1769309802 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658071580 con 0x7f565815db60 2023-12-12T09:52:11.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.258+0000 7f5656ffd640 1 -- 172.21.15.72:0/1769309802 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164482409 0 0) 0x7f5658070fc0 con 0x7f5658159ed0 2023-12-12T09:52:11.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.258+0000 7f565cefe640 1 --2- 172.21.15.72:0/1769309802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56580cc980 0x7f56580ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:11.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.259+0000 7f56577fe640 1 --2- 172.21.15.72:0/1769309802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f565815db60 0x7f565815dff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f564c009870 tx=0x7f564c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e32f3728d87a7b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:11.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.259+0000 7f5656ffd640 1 -- 172.21.15.72:0/1769309802 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c030840 con 0x7f565815db60 2023-12-12T09:52:11.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.259+0000 7f5656ffd640 1 -- 172.21.15.72:0/1769309802 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c0309a0 con 0x7f565815db60 2023-12-12T09:52:11.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.259+0000 7f5656ffd640 1 -- 172.21.15.72:0/1769309802 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c030d30 con 0x7f565815db60 2023-12-12T09:52:11.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.259+0000 7f565e961640 1 -- 172.21.15.72:0/1769309802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f565815db60 msgr2=0x7f565815dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:11.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.259+0000 7f565e961640 1 --2- 172.21.15.72:0/1769309802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f565815db60 0x7f565815dff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f564c009870 tx=0x7f564c02b980 comp rx=0 tx=0).stop 2023-12-12T09:52:11.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.260+0000 7f565e961640 1 -- 172.21.15.72:0/1769309802 shutdown_connections 2023-12-12T09:52:11.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.260+0000 7f565e961640 1 --2- 172.21.15.72:0/1769309802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56580cc980 0x7f56580ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.260+0000 7f565e961640 1 --2- 172.21.15.72:0/1769309802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f565815db60 0x7f565815dff0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.260+0000 7f565e961640 1 -- 172.21.15.72:0/1769309802 >> 172.21.15.72:0/1769309802 conn(0x7f56580b0a90 msgr2=0x7f56580b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:11.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.260+0000 7f565e961640 1 -- 172.21.15.72:0/1769309802 shutdown_connections 2023-12-12T09:52:11.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.260+0000 7f565e961640 1 -- 172.21.15.72:0/1769309802 wait complete. 2023-12-12T09:52:11.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.261+0000 7f565e961640 1 Processor -- start 2023-12-12T09:52:11.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.262+0000 7f565e961640 1 -- start start 2023-12-12T09:52:11.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f565e961640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56580cc980 0x7f56581f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f565e961640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5658159ed0 0x7f56580a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f565e961640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f565815db60 0x7f56580b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f565e961640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580a84f0 con 0x7f56580cc980 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f565e961640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580a87d0 con 0x7f565815db60 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f565e961640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56580a8ab0 con 0x7f5658159ed0 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56580cc980 0x7f56581f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56580cc980 0x7f56581f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40186/0 (socket says 172.21.15.72:40186) 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 -- 172.21.15.72:0/1056196456 learned_addr learned my addr 172.21.15.72:0/1056196456 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 -- 172.21.15.72:0/1056196456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5658159ed0 msgr2=0x7f56580a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5658159ed0 0x7f56580a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 -- 172.21.15.72:0/1056196456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f565815db60 msgr2=0x7f56580b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:11.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f565815db60 0x7f56580b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:11.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 -- 172.21.15.72:0/1056196456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c0094d0 con 0x7f56580cc980 2023-12-12T09:52:11.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.263+0000 7f5657fff640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56580cc980 0x7f56581f6c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f564800c4b0 tx=0x7f564800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:11.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.264+0000 7f5654ff9640 1 -- 172.21.15.72:0/1056196456 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648005480 con 0x7f56580cc980 2023-12-12T09:52:11.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.264+0000 7f5654ff9640 1 -- 172.21.15.72:0/1056196456 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56480035f0 con 0x7f56580cc980 2023-12-12T09:52:11.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.264+0000 7f5654ff9640 1 -- 172.21.15.72:0/1056196456 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56480039e0 con 0x7f56580cc980 2023-12-12T09:52:11.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.264+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56581f3cd0 con 0x7f56580cc980 2023-12-12T09:52:11.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.264+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56580a9320 con 0x7f56580cc980 2023-12-12T09:52:11.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.265+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56581f41f0 con 0x7f56580cc980 2023-12-12T09:52:11.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.266+0000 7f5654ff9640 1 -- 172.21.15.72:0/1056196456 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5648003b40 con 0x7f56580cc980 2023-12-12T09:52:11.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.267+0000 7f5654ff9640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f564406ae40 0x7f564406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:11.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.267+0000 7f5654ff9640 1 -- 172.21.15.72:0/1056196456 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f5648088ed0 con 0x7f56580cc980 2023-12-12T09:52:11.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.268+0000 7f56577fe640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f564406ae40 0x7f564406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:11.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.268+0000 7f56577fe640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f564406ae40 0x7f564406d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f564c036000 tx=0x7f564c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:11.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.273+0000 7f5654ff9640 1 -- 172.21.15.72:0/1056196456 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56581f41f0 con 0x7f56580cc980 2023-12-12T09:52:11.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:11.460+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f56581f41f0 con 0x7f56580cc980 2023-12-12T09:52:11.821 INFO:tasks.ceph.mds.c.smithi072.stderr:2023-12-12T09:52:11.819+0000 7f43d3e15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-12T09:52:12.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.043+0000 7f5654ff9640 1 -- 172.21.15.72:0/1056196456 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f56581f41f0 con 0x7f56580cc980 2023-12-12T09:52:12.046 INFO:teuthology.orchestra.run.smithi072.stderr:set pool 4 allow_ec_overwrites to true 2023-12-12T09:52:12.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f564406ae40 msgr2=0x7f564406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:12.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f564406ae40 0x7f564406d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f564c036000 tx=0x7f564c030000 comp rx=0 tx=0).stop 2023-12-12T09:52:12.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56580cc980 msgr2=0x7f56581f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:12.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56580cc980 0x7f56581f6c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f564800c4b0 tx=0x7f564800c970 comp rx=0 tx=0).stop 2023-12-12T09:52:12.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 shutdown_connections 2023-12-12T09:52:12.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f564406ae40 0x7f564406d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f565815db60 0x7f56580b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5658159ed0 0x7f56580a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 --2- 172.21.15.72:0/1056196456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56580cc980 0x7f56581f6c20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.048+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 >> 172.21.15.72:0/1056196456 conn(0x7f56580b0a90 msgr2=0x7f565806d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:12.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.050+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 shutdown_connections 2023-12-12T09:52:12.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.051+0000 7f565e961640 1 -- 172.21.15.72:0/1056196456 wait complete. 2023-12-12T09:52:12.089 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-12-12T09:52:12.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.280+0000 7f93ffd4c640 1 Processor -- start 2023-12-12T09:52:12.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.280+0000 7f93ffd4c640 1 -- start start 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93ffd4c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f93f806d770 0x7f93f80b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93ffd4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93f806db50 0x7f93f8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93ffd4c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f815b780 con 0x7f93f8141790 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93ffd4c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f8141c20 con 0x7f93f806d770 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93ffd4c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f815b9d0 con 0x7f93f806db50 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93fdae8640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f93f806d770 0x7f93f80b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93fdae8640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f93f806d770 0x7f93f80b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40190/0 (socket says 172.21.15.72:40190) 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93fe2e9640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f93f8141790 0x7f93f815ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:34890/0 (socket says 172.21.15.72:34890) 2023-12-12T09:52:12.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.282+0000 7f93fdae8640 1 -- 172.21.15.72:0/1625064592 learned_addr learned my addr 172.21.15.72:0/1625064592 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:12.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.283+0000 7f93fd2e7640 1 --2- 172.21.15.72:0/1625064592 >> v2:172.21.15.113:3300/0 conn(0x7f93f806db50 0x7f93f8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:12.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.283+0000 7f93fdae8640 1 -- 172.21.15.72:0/1625064592 >> v2:172.21.15.113:3300/0 conn(0x7f93f806db50 msgr2=0x7f93f8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:12.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.283+0000 7f93fdae8640 1 --2- 172.21.15.72:0/1625064592 >> v2:172.21.15.113:3300/0 conn(0x7f93f806db50 0x7f93f8141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.283+0000 7f93fdae8640 1 -- 172.21.15.72:0/1625064592 >> v1:172.21.15.72:6789/0 conn(0x7f93f8141790 legacy=0x7f93f815ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:12.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.283+0000 7f93fdae8640 1 -- 172.21.15.72:0/1625064592 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f815bcb0 con 0x7f93f806d770 2023-12-12T09:52:12.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.284+0000 7f93fdae8640 1 --2- 172.21.15.72:0/1625064592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f806d770 0x7f93f80b1e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93f8067c70 tx=0x7f93f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cc3791af783bf6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.284+0000 7f93fcae6640 1 -- 172.21.15.72:0/1625064592 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4030860 con 0x7f93f806d770 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.284+0000 7f93fcae6640 1 -- 172.21.15.72:0/1625064592 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f40309c0 con 0x7f93f806d770 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.284+0000 7f93fcae6640 1 -- 172.21.15.72:0/1625064592 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4030d50 con 0x7f93f806d770 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.284+0000 7f93ffd4c640 1 -- 172.21.15.72:0/1625064592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f806d770 msgr2=0x7f93f80b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.284+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/1625064592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f806d770 0x7f93f80b1e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93f8067c70 tx=0x7f93f402b9a0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.285+0000 7f93ffd4c640 1 -- 172.21.15.72:0/1625064592 shutdown_connections 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.285+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/1625064592 >> v2:172.21.15.113:3300/0 conn(0x7f93f806db50 0x7f93f8141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.285+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/1625064592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f806d770 0x7f93f80b1e00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.285+0000 7f93ffd4c640 1 -- 172.21.15.72:0/1625064592 >> 172.21.15.72:0/1625064592 conn(0x7f93f80cb850 msgr2=0x7f93f80cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.285+0000 7f93ffd4c640 1 -- 172.21.15.72:0/1625064592 shutdown_connections 2023-12-12T09:52:12.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.285+0000 7f93ffd4c640 1 -- 172.21.15.72:0/1625064592 wait complete. 2023-12-12T09:52:12.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.286+0000 7f93ffd4c640 1 Processor -- start 2023-12-12T09:52:12.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.287+0000 7f93ffd4c640 1 -- start start 2023-12-12T09:52:12.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93ffd4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93f806d770 0x7f93f81e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:12.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93ffd4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93f806db50 0x7f93f81e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:12.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93ffd4c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f8141790 0x7f93f8148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:12.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93ffd4c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f81e7fa0 con 0x7f93f8141790 2023-12-12T09:52:12.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93ffd4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f81476c0 con 0x7f93f806d770 2023-12-12T09:52:12.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93ffd4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f81478d0 con 0x7f93f806db50 2023-12-12T09:52:12.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fe2e9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f8141790 0x7f93f8148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:12.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fe2e9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f8141790 0x7f93f8148690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40196/0 (socket says 172.21.15.72:40196) 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fe2e9640 1 -- 172.21.15.72:0/674828861 learned_addr learned my addr 172.21.15.72:0/674828861 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fe2e9640 1 -- 172.21.15.72:0/674828861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93f806db50 msgr2=0x7f93f81e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fe2e9640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93f806db50 0x7f93f81e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fe2e9640 1 -- 172.21.15.72:0/674828861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93f806d770 msgr2=0x7f93f81e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fdae8640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93f806d770 0x7f93f81e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fe2e9640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93f806d770 0x7f93f81e7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fe2e9640 1 -- 172.21.15.72:0/674828861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f40094d0 con 0x7f93f8141790 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.288+0000 7f93fdae8640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93f806d770 0x7f93f81e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.289+0000 7f93fe2e9640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f8141790 0x7f93f8148690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f93f000c730 tx=0x7f93f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:12.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.289+0000 7f93ee7fc640 1 -- 172.21.15.72:0/674828861 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000ce40 con 0x7f93f8141790 2023-12-12T09:52:12.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.289+0000 7f93ee7fc640 1 -- 172.21.15.72:0/674828861 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f000a640 con 0x7f93f8141790 2023-12-12T09:52:12.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.289+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f8147bb0 con 0x7f93f8141790 2023-12-12T09:52:12.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.289+0000 7f93ee7fc640 1 -- 172.21.15.72:0/674828861 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000a9f0 con 0x7f93f8141790 2023-12-12T09:52:12.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.289+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f8148170 con 0x7f93f8141790 2023-12-12T09:52:12.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.290+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f8148390 con 0x7f93f8141790 2023-12-12T09:52:12.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.291+0000 7f93ee7fc640 1 -- 172.21.15.72:0/674828861 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93f000ab50 con 0x7f93f8141790 2023-12-12T09:52:12.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.292+0000 7f93ee7fc640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f93dc06ae40 0x7f93dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:12.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.292+0000 7f93ee7fc640 1 -- 172.21.15.72:0/674828861 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f93f0088ff0 con 0x7f93f8141790 2023-12-12T09:52:12.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.292+0000 7f93fdae8640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f93dc06ae40 0x7f93dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:12.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.293+0000 7f93fdae8640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f93dc06ae40 0x7f93dc06d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93f4036000 tx=0x7f93f403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.296+0000 7f93ee7fc640 1 -- 172.21.15.72:0/674828861 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93f00bbe90 con 0x7f93f8141790 2023-12-12T09:52:12.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:12.517+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f93f80745d0 con 0x7f93f8141790 2023-12-12T09:52:13.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.061+0000 7f93ee7fc640 1 -- 172.21.15.72:0/674828861 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f93f0017020 con 0x7f93f8141790 2023-12-12T09:52:13.064 INFO:teuthology.orchestra.run.smithi072.stderr:added data pool 4 to fsmap 2023-12-12T09:52:13.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f93dc06ae40 msgr2=0x7f93dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f93dc06ae40 0x7f93dc06d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93f4036000 tx=0x7f93f403b040 comp rx=0 tx=0).stop 2023-12-12T09:52:13.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f8141790 msgr2=0x7f93f8148690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f8141790 0x7f93f8148690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f93f000c730 tx=0x7f93f000cbf0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 shutdown_connections 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f93dc06ae40 0x7f93dc06d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93f8141790 0x7f93f8148690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93f806db50 0x7f93f81e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 --2- 172.21.15.72:0/674828861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93f806d770 0x7f93f81e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.065+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 >> 172.21.15.72:0/674828861 conn(0x7f93f80cb850 msgr2=0x7f93f813f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.066+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 shutdown_connections 2023-12-12T09:52:13.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.066+0000 7f93ffd4c640 1 -- 172.21.15.72:0/674828861 wait complete. 2023-12-12T09:52:13.087 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:52:13.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.295+0000 7fd83b59e640 1 Processor -- start 2023-12-12T09:52:13.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.295+0000 7fd83b59e640 1 -- start start 2023-12-12T09:52:13.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.297+0000 7fd83b59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06ecd0 0x7fd83c146320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.297+0000 7fd83b59e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd83c146860 0x7fd83c06a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.297+0000 7fd83b59e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c146150 con 0x7fd83c06a9a0 2023-12-12T09:52:13.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.297+0000 7fd83b59e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c06ae50 con 0x7fd83c146860 2023-12-12T09:52:13.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.297+0000 7fd83b59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c06f0b0 con 0x7fd83c06ecd0 2023-12-12T09:52:13.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.297+0000 7fd839d9b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd83c146860 0x7fd83c06a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.297+0000 7fd83ad9d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd83c06a9a0 0x7fd83c160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:34894/0 (socket says 172.21.15.72:34894) 2023-12-12T09:52:13.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.297+0000 7fd83ad9d640 1 -- 172.21.15.72:0/1565453127 learned_addr learned my addr 172.21.15.72:0/1565453127 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:13.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.298+0000 7fd83a59c640 1 --2- 172.21.15.72:0/1565453127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06ecd0 0x7fd83c146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.298+0000 7fd839d9b640 1 -- 172.21.15.72:0/1565453127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06ecd0 msgr2=0x7fd83c146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:13.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.298+0000 7fd839d9b640 1 --2- 172.21.15.72:0/1565453127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06ecd0 0x7fd83c146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.298+0000 7fd839d9b640 1 -- 172.21.15.72:0/1565453127 >> v1:172.21.15.72:6789/0 conn(0x7fd83c06a9a0 legacy=0x7fd83c160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.298+0000 7fd83959a640 1 -- 172.21.15.72:0/1565453127 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985964158 0 0) 0x7fd83c146150 con 0x7fd83c06a9a0 2023-12-12T09:52:13.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.298+0000 7fd839d9b640 1 -- 172.21.15.72:0/1565453127 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0d0550 con 0x7fd83c146860 2023-12-12T09:52:13.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.298+0000 7fd83a59c640 1 --2- 172.21.15.72:0/1565453127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06ecd0 0x7fd83c146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:13.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.299+0000 7fd839d9b640 1 --2- 172.21.15.72:0/1565453127 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c06a460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8300097a0 tx=0x7fd83002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9635d12af99950 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:13.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.299+0000 7fd83959a640 1 -- 172.21.15.72:0/1565453127 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830031830 con 0x7fd83c146860 2023-12-12T09:52:13.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.299+0000 7fd83959a640 1 -- 172.21.15.72:0/1565453127 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd830031990 con 0x7fd83c146860 2023-12-12T09:52:13.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.299+0000 7fd83959a640 1 -- 172.21.15.72:0/1565453127 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830031d20 con 0x7fd83c146860 2023-12-12T09:52:13.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.300+0000 7fd83b59e640 1 -- 172.21.15.72:0/1565453127 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 msgr2=0x7fd83c06a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.300+0000 7fd83b59e640 1 --2- 172.21.15.72:0/1565453127 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c06a460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8300097a0 tx=0x7fd83002b8d0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.300+0000 7fd83b59e640 1 -- 172.21.15.72:0/1565453127 shutdown_connections 2023-12-12T09:52:13.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.300+0000 7fd83b59e640 1 --2- 172.21.15.72:0/1565453127 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c06a460 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.300+0000 7fd83b59e640 1 --2- 172.21.15.72:0/1565453127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06ecd0 0x7fd83c146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.300+0000 7fd83b59e640 1 -- 172.21.15.72:0/1565453127 >> 172.21.15.72:0/1565453127 conn(0x7fd83c0b1840 msgr2=0x7fd83c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:13.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.300+0000 7fd83b59e640 1 -- 172.21.15.72:0/1565453127 shutdown_connections 2023-12-12T09:52:13.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.300+0000 7fd83b59e640 1 -- 172.21.15.72:0/1565453127 wait complete. 2023-12-12T09:52:13.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.301+0000 7fd83b59e640 1 Processor -- start 2023-12-12T09:52:13.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.301+0000 7fd83b59e640 1 -- start start 2023-12-12T09:52:13.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.302+0000 7fd83b59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06a9a0 0x7fd83c137620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.302+0000 7fd83b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd83c06ecd0 0x7fd83c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.302+0000 7fd83b59e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.302+0000 7fd83b59e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c15d270 con 0x7fd83c146860 2023-12-12T09:52:13.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.302+0000 7fd83b59e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c141cf0 con 0x7fd83c06ecd0 2023-12-12T09:52:13.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.302+0000 7fd83b59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c06ddf0 con 0x7fd83c06a9a0 2023-12-12T09:52:13.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c0c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40220/0 (socket says 172.21.15.72:40220) 2023-12-12T09:52:13.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 -- 172.21.15.72:0/3452314839 learned_addr learned my addr 172.21.15.72:0/3452314839 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:13.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 -- 172.21.15.72:0/3452314839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06a9a0 msgr2=0x7fd83c137620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06a9a0 0x7fd83c137620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 -- 172.21.15.72:0/3452314839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd83c06ecd0 msgr2=0x7fd83c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd83c06ecd0 0x7fd83c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 -- 172.21.15.72:0/3452314839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd830009400 con 0x7fd83c146860 2023-12-12T09:52:13.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83ad9d640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c0c6800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd83400c800 tx=0x7fd83400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:13.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd82affd640 1 -- 172.21.15.72:0/3452314839 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83400a640 con 0x7fd83c146860 2023-12-12T09:52:13.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd82affd640 1 -- 172.21.15.72:0/3452314839 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83400a7e0 con 0x7fd83c146860 2023-12-12T09:52:13.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.303+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c15f180 con 0x7fd83c146860 2023-12-12T09:52:13.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.304+0000 7fd82affd640 1 -- 172.21.15.72:0/3452314839 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83400ab90 con 0x7fd83c146860 2023-12-12T09:52:13.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.304+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c15f740 con 0x7fd83c146860 2023-12-12T09:52:13.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.305+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c072160 con 0x7fd83c146860 2023-12-12T09:52:13.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.305+0000 7fd82affd640 1 -- 172.21.15.72:0/3452314839 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd83400acf0 con 0x7fd83c146860 2023-12-12T09:52:13.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.306+0000 7fd82affd640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd81c06ae40 0x7fd81c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.306+0000 7fd82affd640 1 -- 172.21.15.72:0/3452314839 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd834088ff0 con 0x7fd83c146860 2023-12-12T09:52:13.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.307+0000 7fd83a59c640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd81c06ae40 0x7fd81c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.307+0000 7fd83a59c640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd81c06ae40 0x7fd81c06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd82c006fd0 tx=0x7fd82c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:13.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.311+0000 7fd82affd640 1 -- 172.21.15.72:0/3452314839 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8340bbe90 con 0x7fd83c146860 2023-12-12T09:52:13.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.538+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd83c0673f0 con 0x7fd83c146860 2023-12-12T09:52:13.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.539+0000 7fd82affd640 1 -- 172.21.15.72:0/3452314839 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fd834018070 con 0x7fd83c146860 2023-12-12T09:52:13.541 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:13.541 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:12.519576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-12T09:52:13.541 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 6 2023-12-12T09:52:13.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.541+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd81c06ae40 msgr2=0x7fd81c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.541+0000 7fd83b59e640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd81c06ae40 0x7fd81c06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd82c006fd0 tx=0x7fd82c008040 comp rx=0 tx=0).stop 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.541+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 msgr2=0x7fd83c0c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.541+0000 7fd83b59e640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c0c6800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd83400c800 tx=0x7fd83400ccc0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.542+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 shutdown_connections 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.542+0000 7fd83b59e640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd81c06ae40 0x7fd81c06d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.542+0000 7fd83b59e640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd83c146860 0x7fd83c0c6800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.542+0000 7fd83b59e640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd83c06ecd0 0x7fd83c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.542+0000 7fd83b59e640 1 --2- 172.21.15.72:0/3452314839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd83c06a9a0 0x7fd83c137620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.542+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 >> 172.21.15.72:0/3452314839 conn(0x7fd83c0b1840 msgr2=0x7fd83c145600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.542+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 shutdown_connections 2023-12-12T09:52:13.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.542+0000 7fd83b59e640 1 -- 172.21.15.72:0/3452314839 wait complete. 2023-12-12T09:52:13.562 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:13.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.767+0000 7f180b223640 1 Processor -- start 2023-12-12T09:52:13.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.767+0000 7f180b223640 1 -- start start 2023-12-12T09:52:13.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.769+0000 7f180b223640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1804157a60 0x7f180406c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.769+0000 7f180b223640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180406de20 0x7f180415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.769+0000 7f180b223640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180406ca40 con 0x7f180415e0e0 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.769+0000 7f180b223640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180415e550 con 0x7f1804157a60 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.769+0000 7f180b223640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1804071cd0 con 0x7f180406de20 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.769+0000 7f1808fbf640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1804157a60 0x7f180406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.769+0000 7f1808fbf640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1804157a60 0x7f180406c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40222/0 (socket says 172.21.15.72:40222) 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.769+0000 7f1808fbf640 1 -- 172.21.15.72:0/3156167585 learned_addr learned my addr 172.21.15.72:0/3156167585 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.770+0000 7f1803fff640 1 --2- 172.21.15.72:0/3156167585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180406de20 0x7f180415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.770+0000 7f1808fbf640 1 -- 172.21.15.72:0/3156167585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180406de20 msgr2=0x7f180415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.770+0000 7f1808fbf640 1 --2- 172.21.15.72:0/3156167585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180406de20 0x7f180415dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.770+0000 7f1808fbf640 1 -- 172.21.15.72:0/3156167585 >> v1:172.21.15.113:6789/0 conn(0x7f180415e0e0 legacy=0x7f180415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:13.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.770+0000 7f1808fbf640 1 -- 172.21.15.72:0/3156167585 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804070f10 con 0x7f1804157a60 2023-12-12T09:52:13.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.771+0000 7f1808fbf640 1 --2- 172.21.15.72:0/3156167585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1804157a60 0x7f180406c410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f17f800bb10 tx=0x7f17f802e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9403e6c148381bce server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:13.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.771+0000 7f18037fe640 1 -- 172.21.15.72:0/3156167585 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f802ede0 con 0x7f1804157a60 2023-12-12T09:52:13.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.771+0000 7f18037fe640 1 -- 172.21.15.72:0/3156167585 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f8036440 con 0x7f1804157a60 2023-12-12T09:52:13.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.771+0000 7f18037fe640 1 -- 172.21.15.72:0/3156167585 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f80367d0 con 0x7f1804157a60 2023-12-12T09:52:13.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.772+0000 7f180b223640 1 -- 172.21.15.72:0/3156167585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1804157a60 msgr2=0x7f180406c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.772+0000 7f180b223640 1 --2- 172.21.15.72:0/3156167585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1804157a60 0x7f180406c410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f17f800bb10 tx=0x7f17f802e030 comp rx=0 tx=0).stop 2023-12-12T09:52:13.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.772+0000 7f180b223640 1 -- 172.21.15.72:0/3156167585 shutdown_connections 2023-12-12T09:52:13.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.772+0000 7f180b223640 1 --2- 172.21.15.72:0/3156167585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180406de20 0x7f180415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.772+0000 7f180b223640 1 --2- 172.21.15.72:0/3156167585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1804157a60 0x7f180406c410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.772+0000 7f180b223640 1 -- 172.21.15.72:0/3156167585 >> 172.21.15.72:0/3156167585 conn(0x7f18040b0a90 msgr2=0x7f18040b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:13.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.772+0000 7f180b223640 1 -- 172.21.15.72:0/3156167585 shutdown_connections 2023-12-12T09:52:13.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.772+0000 7f180b223640 1 -- 172.21.15.72:0/3156167585 wait complete. 2023-12-12T09:52:13.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.774+0000 7f180b223640 1 Processor -- start 2023-12-12T09:52:13.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.774+0000 7f180b223640 1 -- start start 2023-12-12T09:52:13.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f180b223640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f180406de20 0x7f180414b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f180b223640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1804157a60 0x7f18040b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f180b223640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180415e0e0 0x7f180413f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f180b223640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18040b2f40 con 0x7f180406de20 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f180b223640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18040b3200 con 0x7f1804157a60 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f180b223640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18040b34e0 con 0x7f180415e0e0 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f1808fbf640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f180406de20 0x7f180414b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f1808fbf640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f180406de20 0x7f180414b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40226/0 (socket says 172.21.15.72:40226) 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.775+0000 7f1808fbf640 1 -- 172.21.15.72:0/1452144903 learned_addr learned my addr 172.21.15.72:0/1452144903 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f1808fbf640 1 -- 172.21.15.72:0/1452144903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180415e0e0 msgr2=0x7f180413f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f18097c0640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180415e0e0 0x7f180413f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f1803fff640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1804157a60 0x7f18040b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f1808fbf640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180415e0e0 0x7f180413f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f1808fbf640 1 -- 172.21.15.72:0/1452144903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1804157a60 msgr2=0x7f18040b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f1808fbf640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1804157a60 0x7f18040b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f1808fbf640 1 -- 172.21.15.72:0/1452144903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f800b770 con 0x7f180406de20 2023-12-12T09:52:13.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f1808fbf640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f180406de20 0x7f180414b590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17f802e070 tx=0x7f17f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:13.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f18017fa640 1 -- 172.21.15.72:0/1452144903 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f803f840 con 0x7f180406de20 2023-12-12T09:52:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f18017fa640 1 -- 172.21.15.72:0/1452144903 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f803f9a0 con 0x7f180406de20 2023-12-12T09:52:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f18017fa640 1 -- 172.21.15.72:0/1452144903 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f803fd50 con 0x7f180406de20 2023-12-12T09:52:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1804152c40 con 0x7f180406de20 2023-12-12T09:52:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.776+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180413fc20 con 0x7f180406de20 2023-12-12T09:52:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.778+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1804152aa0 con 0x7f180406de20 2023-12-12T09:52:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.778+0000 7f18017fa640 1 -- 172.21.15.72:0/1452144903 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17f803d040 con 0x7f180406de20 2023-12-12T09:52:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.779+0000 7f18017fa640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f17e806ae40 0x7f17e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.779+0000 7f18017fa640 1 -- 172.21.15.72:0/1452144903 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f17f80ae300 con 0x7f180406de20 2023-12-12T09:52:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.780+0000 7f1803fff640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f17e806ae40 0x7f17e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.780+0000 7f1803fff640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f17e806ae40 0x7f17e806d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f17f0006fd0 tx=0x7f17f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:13.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.784+0000 7f18017fa640 1 -- 172.21.15.72:0/1452144903 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1804152aa0 con 0x7f180406de20 2023-12-12T09:52:13.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.974+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1804152aa0 con 0x7f180406de20 2023-12-12T09:52:13.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.978+0000 7f18017fa640 1 -- 172.21.15.72:0/1452144903 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f1804152aa0 con 0x7f180406de20 2023-12-12T09:52:13.980 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:13.980 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.980+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f17e806ae40 msgr2=0x7f17e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.980+0000 7f180b223640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f17e806ae40 0x7f17e806d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f17f0006fd0 tx=0x7f17f0008040 comp rx=0 tx=0).stop 2023-12-12T09:52:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.980+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f180406de20 msgr2=0x7f180414b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.980+0000 7f180b223640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f180406de20 0x7f180414b590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17f802e070 tx=0x7f17f80046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.981+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 shutdown_connections 2023-12-12T09:52:13.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.981+0000 7f180b223640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f17e806ae40 0x7f17e806d300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.981+0000 7f180b223640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f180415e0e0 0x7f180413f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.981+0000 7f180b223640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1804157a60 0x7f18040b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.981+0000 7f180b223640 1 --2- 172.21.15.72:0/1452144903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f180406de20 0x7f180414b590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:13.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.981+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 >> 172.21.15.72:0/1452144903 conn(0x7f18040b0a90 msgr2=0x7f18040701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:13.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.981+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 shutdown_connections 2023-12-12T09:52:13.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:13.981+0000 7f180b223640 1 -- 172.21.15.72:0/1452144903 wait complete. 2023-12-12T09:52:14.001 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:14.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.204+0000 7f79feeb4640 1 Processor -- start 2023-12-12T09:52:14.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.205+0000 7f79feeb4640 1 -- start start 2023-12-12T09:52:14.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.206+0000 7f79feeb4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79f80d1b20 0x7f79f80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.206+0000 7f79feeb4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f80b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.206+0000 7f79feeb4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f80b0a00 con 0x7f79f815d390 2023-12-12T09:52:14.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.206+0000 7f79feeb4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f80ceea0 con 0x7f79f80d1b20 2023-12-12T09:52:14.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.206+0000 7f79feeb4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f815d770 con 0x7f79f80d0c10 2023-12-12T09:52:14.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fe6b3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f80b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fd6b1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79f80d1b20 0x7f79f80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fd6b1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79f80d1b20 0x7f79f80b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:51440/0 (socket says 172.21.15.72:51440) 2023-12-12T09:52:14.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fd6b1640 1 -- 172.21.15.72:0/2367944180 learned_addr learned my addr 172.21.15.72:0/2367944180 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:14.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fe6b3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f80b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:55152/0 (socket says 172.21.15.72:55152) 2023-12-12T09:52:14.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fe6b3640 1 -- 172.21.15.72:0/2367944180 >> v2:172.21.15.113:3300/0 conn(0x7f79f80d1b20 msgr2=0x7f79f80b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:14.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fe6b3640 1 --2- 172.21.15.72:0/2367944180 >> v2:172.21.15.113:3300/0 conn(0x7f79f80d1b20 0x7f79f80b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fe6b3640 1 -- 172.21.15.72:0/2367944180 >> v1:172.21.15.113:6789/0 conn(0x7f79f815d390 legacy=0x7f79f80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.207+0000 7f79fe6b3640 1 -- 172.21.15.72:0/2367944180 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f80b1840 con 0x7f79f80d0c10 2023-12-12T09:52:14.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.209+0000 7f79fe6b3640 1 --2- 172.21.15.72:0/2367944180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f80b04c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f79f00097a0 tx=0x7f79f002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cbe2f3861255808 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:14.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79fceb0640 1 -- 172.21.15.72:0/2367944180 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00316f0 con 0x7f79f80d0c10 2023-12-12T09:52:14.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79fceb0640 1 -- 172.21.15.72:0/2367944180 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0031850 con 0x7f79f80d0c10 2023-12-12T09:52:14.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79fceb0640 1 -- 172.21.15.72:0/2367944180 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0031be0 con 0x7f79f80d0c10 2023-12-12T09:52:14.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79feeb4640 1 -- 172.21.15.72:0/2367944180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 msgr2=0x7f79f80b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79feeb4640 1 --2- 172.21.15.72:0/2367944180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f80b04c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f79f00097a0 tx=0x7f79f002b790 comp rx=0 tx=0).stop 2023-12-12T09:52:14.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79feeb4640 1 -- 172.21.15.72:0/2367944180 shutdown_connections 2023-12-12T09:52:14.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79feeb4640 1 --2- 172.21.15.72:0/2367944180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f80b04c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79feeb4640 1 --2- 172.21.15.72:0/2367944180 >> v2:172.21.15.113:3300/0 conn(0x7f79f80d1b20 0x7f79f80b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.210+0000 7f79feeb4640 1 -- 172.21.15.72:0/2367944180 >> 172.21.15.72:0/2367944180 conn(0x7f79f80d2ec0 msgr2=0x7f79f80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:14.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.211+0000 7f79feeb4640 1 -- 172.21.15.72:0/2367944180 shutdown_connections 2023-12-12T09:52:14.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.211+0000 7f79feeb4640 1 -- 172.21.15.72:0/2367944180 wait complete. 2023-12-12T09:52:14.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.212+0000 7f79feeb4640 1 Processor -- start 2023-12-12T09:52:14.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.212+0000 7f79feeb4640 1 -- start start 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.213+0000 7f79feeb4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f809dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.213+0000 7f79feeb4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80d1b20 0x7f79f80b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.213+0000 7f79feeb4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f815d390 0x7f79f815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.213+0000 7f79feeb4640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f8144d30 con 0x7f79f815d390 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.213+0000 7f79feeb4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f8144120 con 0x7f79f80d1b20 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.213+0000 7f79feeb4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f8144400 con 0x7f79f80d0c10 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f815d390 0x7f79f815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f815d390 0x7f79f815a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40230/0 (socket says 172.21.15.72:40230) 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 -- 172.21.15.72:0/1577700248 learned_addr learned my addr 172.21.15.72:0/1577700248 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fdeb2640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f809dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fd6b1640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80d1b20 0x7f79f80b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 -- 172.21.15.72:0/1577700248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 msgr2=0x7f79f809dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f809dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 -- 172.21.15.72:0/1577700248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80d1b20 msgr2=0x7f79f80b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80d1b20 0x7f79f80b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 -- 172.21.15.72:0/1577700248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009400 con 0x7f79f815d390 2023-12-12T09:52:14.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.214+0000 7f79fe6b3640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f815d390 0x7f79f815a100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79f00097a0 tx=0x7f79f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.215+0000 7f79ea7fc640 1 -- 172.21.15.72:0/1577700248 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0034680 con 0x7f79f815d390 2023-12-12T09:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.215+0000 7f79ea7fc640 1 -- 172.21.15.72:0/1577700248 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f00347e0 con 0x7f79f815d390 2023-12-12T09:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.215+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f81446e0 con 0x7f79f815d390 2023-12-12T09:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.215+0000 7f79ea7fc640 1 -- 172.21.15.72:0/1577700248 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0034b70 con 0x7f79f815d390 2023-12-12T09:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.215+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f80b27e0 con 0x7f79f815d390 2023-12-12T09:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.216+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f80b2970 con 0x7f79f815d390 2023-12-12T09:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.217+0000 7f79ea7fc640 1 -- 172.21.15.72:0/1577700248 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79f003b040 con 0x7f79f815d390 2023-12-12T09:52:14.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.218+0000 7f79ea7fc640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f79dc06ae40 0x7f79dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.218+0000 7f79ea7fc640 1 -- 172.21.15.72:0/1577700248 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f79f00aaea0 con 0x7f79f815d390 2023-12-12T09:52:14.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.218+0000 7f79fdeb2640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f79dc06ae40 0x7f79dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.219+0000 7f79fdeb2640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f79dc06ae40 0x7f79dc06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79f81f62f0 tx=0x7f79f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:14.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.222+0000 7f79ea7fc640 1 -- 172.21.15.72:0/1577700248 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79f80b2970 con 0x7f79f815d390 2023-12-12T09:52:14.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.416+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79f80b2970 con 0x7f79f815d390 2023-12-12T09:52:14.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.418+0000 7f79ea7fc640 1 -- 172.21.15.72:0/1577700248 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f79f80b2970 con 0x7f79f815d390 2023-12-12T09:52:14.420 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:14.420 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:14.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f79dc06ae40 msgr2=0x7f79dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f79dc06ae40 0x7f79dc06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79f81f62f0 tx=0x7f79f4009040 comp rx=0 tx=0).stop 2023-12-12T09:52:14.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f815d390 msgr2=0x7f79f815a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f815d390 0x7f79f815a100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79f00097a0 tx=0x7f79f00046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 shutdown_connections 2023-12-12T09:52:14.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f79dc06ae40 0x7f79dc06d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f815d390 0x7f79f815a100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80d1b20 0x7f79f80b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 --2- 172.21.15.72:0/1577700248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80d0c10 0x7f79f809dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 >> 172.21.15.72:0/1577700248 conn(0x7f79f80d2ec0 msgr2=0x7f79f80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:14.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.421+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 shutdown_connections 2023-12-12T09:52:14.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.422+0000 7f79feeb4640 1 -- 172.21.15.72:0/1577700248 wait complete. 2023-12-12T09:52:14.464 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:52:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.662+0000 7f981e57c640 1 Processor -- start 2023-12-12T09:52:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.663+0000 7f981e57c640 1 -- start start 2023-12-12T09:52:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.664+0000 7f981e57c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f981815d390 0x7f98180b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.664+0000 7f981e57c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98180b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.664+0000 7f981e57c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98180b0a00 con 0x7f98180d1410 2023-12-12T09:52:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.664+0000 7f981e57c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98180ceea0 con 0x7f981815d390 2023-12-12T09:52:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.664+0000 7f981e57c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981815d770 con 0x7f98180d0c10 2023-12-12T09:52:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.664+0000 7f981d57a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f981815d390 0x7f98180b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.664+0000 7f981d57a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f981815d390 0x7f98180b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40236/0 (socket says 172.21.15.72:40236) 2023-12-12T09:52:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.664+0000 7f981d57a640 1 -- 172.21.15.72:0/126989557 learned_addr learned my addr 172.21.15.72:0/126989557 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.665+0000 7f981cd79640 1 --2- 172.21.15.72:0/126989557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98180b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.665+0000 7f981d57a640 1 -- 172.21.15.72:0/126989557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 msgr2=0x7f98180b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.665+0000 7f981d57a640 1 --2- 172.21.15.72:0/126989557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98180b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.665+0000 7f981d57a640 1 -- 172.21.15.72:0/126989557 >> v1:172.21.15.72:6789/0 conn(0x7f98180d1410 legacy=0x7f98180b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.665+0000 7f980ffff640 1 -- 172.21.15.72:0/126989557 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999564971 0 0) 0x7f98180b0a00 con 0x7f98180d1410 2023-12-12T09:52:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.665+0000 7f981d57a640 1 -- 172.21.15.72:0/126989557 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180b1840 con 0x7f981815d390 2023-12-12T09:52:14.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.665+0000 7f981cd79640 1 --2- 172.21.15.72:0/126989557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98180b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.666+0000 7f981d57a640 1 --2- 172.21.15.72:0/126989557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f98180b1cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9814009870 tx=0x7f981402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc8e29bf408054e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.666+0000 7f980ffff640 1 -- 172.21.15.72:0/126989557 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814031640 con 0x7f981815d390 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.666+0000 7f980ffff640 1 -- 172.21.15.72:0/126989557 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140317a0 con 0x7f981815d390 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.666+0000 7f980ffff640 1 -- 172.21.15.72:0/126989557 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814031b30 con 0x7f981815d390 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.667+0000 7f981e57c640 1 -- 172.21.15.72:0/126989557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 msgr2=0x7f98180b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.667+0000 7f981e57c640 1 --2- 172.21.15.72:0/126989557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f98180b1cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9814009870 tx=0x7f981402b8f0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.667+0000 7f981e57c640 1 -- 172.21.15.72:0/126989557 shutdown_connections 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.667+0000 7f981e57c640 1 --2- 172.21.15.72:0/126989557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98180b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.667+0000 7f981e57c640 1 --2- 172.21.15.72:0/126989557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f98180b1cd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.667+0000 7f981e57c640 1 -- 172.21.15.72:0/126989557 >> 172.21.15.72:0/126989557 conn(0x7f98180d2ec0 msgr2=0x7f98180d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.667+0000 7f981e57c640 1 -- 172.21.15.72:0/126989557 shutdown_connections 2023-12-12T09:52:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.667+0000 7f981e57c640 1 -- 172.21.15.72:0/126989557 wait complete. 2023-12-12T09:52:14.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.669+0000 7f981e57c640 1 Processor -- start 2023-12-12T09:52:14.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.669+0000 7f981e57c640 1 -- start start 2023-12-12T09:52:14.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981e57c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98181524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981e57c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98180d1410 0x7f98180b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981e57c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f9818152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981e57c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981809fe30 con 0x7f981815d390 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981e57c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180b38e0 con 0x7f98180d1410 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981e57c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981809f1b0 con 0x7f98180d0c10 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981dd7b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f9818152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981dd7b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f9818152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40244/0 (socket says 172.21.15.72:40244) 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981dd7b640 1 -- 172.21.15.72:0/3481455169 learned_addr learned my addr 172.21.15.72:0/3481455169 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.670+0000 7f981d57a640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98181524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981cd79640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98180d1410 0x7f98180b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981dd7b640 1 -- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 msgr2=0x7f98181524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981dd7b640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98181524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981dd7b640 1 -- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98180d1410 msgr2=0x7f98180b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981dd7b640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98180d1410 0x7f98180b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981dd7b640 1 -- 172.21.15.72:0/3481455169 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98140094d0 con 0x7f981815d390 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981d57a640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98181524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981cd79640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98180d1410 0x7f98180b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981dd7b640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f9818152b70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9810001560 tx=0x7f981000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:14.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f980dffb640 1 -- 172.21.15.72:0/3481455169 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810003da0 con 0x7f981815d390 2023-12-12T09:52:14.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f980dffb640 1 -- 172.21.15.72:0/3481455169 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9810005070 con 0x7f981815d390 2023-12-12T09:52:14.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981809f490 con 0x7f981815d390 2023-12-12T09:52:14.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f980dffb640 1 -- 172.21.15.72:0/3481455169 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810005420 con 0x7f981815d390 2023-12-12T09:52:14.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.671+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981809fa50 con 0x7f981815d390 2023-12-12T09:52:14.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.672+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981809ff70 con 0x7f981815d390 2023-12-12T09:52:14.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.673+0000 7f980dffb640 1 -- 172.21.15.72:0/3481455169 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9810005580 con 0x7f981815d390 2023-12-12T09:52:14.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.674+0000 7f980dffb640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f97fc06ae40 0x7f97fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:14.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.674+0000 7f980dffb640 1 -- 172.21.15.72:0/3481455169 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9810089220 con 0x7f981815d390 2023-12-12T09:52:14.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.675+0000 7f981d57a640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f97fc06ae40 0x7f97fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:14.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.675+0000 7f981d57a640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f97fc06ae40 0x7f97fc06d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9814036000 tx=0x7f981403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:14.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.679+0000 7f980dffb640 1 -- 172.21.15.72:0/3481455169 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f981809ff70 con 0x7f981815d390 2023-12-12T09:52:14.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.905+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f981809ff70 con 0x7f981815d390 2023-12-12T09:52:14.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.906+0000 7f980dffb640 1 -- 172.21.15.72:0/3481455169 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f981809ff70 con 0x7f981815d390 2023-12-12T09:52:14.908 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:14.908 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:12.519576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-12T09:52:14.908 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 6 2023-12-12T09:52:14.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f97fc06ae40 msgr2=0x7f97fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f97fc06ae40 0x7f97fc06d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9814036000 tx=0x7f981403c040 comp rx=0 tx=0).stop 2023-12-12T09:52:14.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 msgr2=0x7f9818152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:14.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f9818152b70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9810001560 tx=0x7f981000cc70 comp rx=0 tx=0).stop 2023-12-12T09:52:14.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 shutdown_connections 2023-12-12T09:52:14.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f97fc06ae40 0x7f97fc06d300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981815d390 0x7f9818152b70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98180d1410 0x7f98180b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 --2- 172.21.15.72:0/3481455169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98180d0c10 0x7f98181524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:14.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 >> 172.21.15.72:0/3481455169 conn(0x7f98180d2ec0 msgr2=0x7f98180cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:14.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 shutdown_connections 2023-12-12T09:52:14.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:14.908+0000 7f981e57c640 1 -- 172.21.15.72:0/3481455169 wait complete. 2023-12-12T09:52:14.928 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:15.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.132+0000 7f742e343640 1 Processor -- start 2023-12-12T09:52:15.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.132+0000 7f742e343640 1 -- start start 2023-12-12T09:52:15.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f742e343640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7428143e00 0x7f74280b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f742e343640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7428072180 0x7f74280b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f742e343640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74280b3bb0 con 0x7f742806c410 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f742e343640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74281441e0 con 0x7f7428072180 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f742e343640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7428072680 con 0x7f7428143e00 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f741f7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7428072180 0x7f74280b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f741f7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7428072180 0x7f74280b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34216/0 (socket says 172.21.15.72:34216) 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f741f7fe640 1 -- 172.21.15.72:0/114402657 learned_addr learned my addr 172.21.15.72:0/114402657 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.134+0000 7f7427fff640 1 --2- 172.21.15.72:0/114402657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7428143e00 0x7f74280b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.135+0000 7f741f7fe640 1 -- 172.21.15.72:0/114402657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7428143e00 msgr2=0x7f74280b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.135+0000 7f741f7fe640 1 --2- 172.21.15.72:0/114402657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7428143e00 0x7f74280b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.135+0000 7f741f7fe640 1 -- 172.21.15.72:0/114402657 >> v1:172.21.15.113:6789/0 conn(0x7f742806c410 legacy=0x7f7428070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:15.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.135+0000 7f741f7fe640 1 -- 172.21.15.72:0/114402657 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280b1e30 con 0x7f7428072180 2023-12-12T09:52:15.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.136+0000 7f741f7fe640 1 --2- 172.21.15.72:0/114402657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f74280b4580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7410009870 tx=0x7f741002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c7118e6987b927 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:15.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.136+0000 7f74277fe640 1 -- 172.21.15.72:0/114402657 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410031640 con 0x7f7428072180 2023-12-12T09:52:15.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.136+0000 7f74277fe640 1 -- 172.21.15.72:0/114402657 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74100317a0 con 0x7f7428072180 2023-12-12T09:52:15.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.136+0000 7f74277fe640 1 -- 172.21.15.72:0/114402657 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410031b30 con 0x7f7428072180 2023-12-12T09:52:15.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.137+0000 7f742e343640 1 -- 172.21.15.72:0/114402657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 msgr2=0x7f74280b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.137+0000 7f742e343640 1 --2- 172.21.15.72:0/114402657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f74280b4580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7410009870 tx=0x7f741002b8f0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.137+0000 7f742e343640 1 -- 172.21.15.72:0/114402657 shutdown_connections 2023-12-12T09:52:15.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.137+0000 7f742e343640 1 --2- 172.21.15.72:0/114402657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f74280b4580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.137+0000 7f742e343640 1 --2- 172.21.15.72:0/114402657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7428143e00 0x7f74280b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.137+0000 7f742e343640 1 -- 172.21.15.72:0/114402657 >> 172.21.15.72:0/114402657 conn(0x7f74280b0a90 msgr2=0x7f74280b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:15.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.137+0000 7f742e343640 1 -- 172.21.15.72:0/114402657 shutdown_connections 2023-12-12T09:52:15.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.137+0000 7f742e343640 1 -- 172.21.15.72:0/114402657 wait complete. 2023-12-12T09:52:15.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.139+0000 7f742e343640 1 Processor -- start 2023-12-12T09:52:15.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.139+0000 7f742e343640 1 -- start start 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f742e343640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f742806c410 0x7f742815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f742e343640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f742813b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f742e343640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7428143e00 0x7f74280cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f742e343640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742813a6f0 con 0x7f7428072180 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f742e343640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742813a9d0 con 0x7f7428143e00 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f742e343640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742813acb0 con 0x7f742806c410 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f741f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f742813b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f741f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f742813b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34232/0 (socket says 172.21.15.72:34232) 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f741f7fe640 1 -- 172.21.15.72:0/1603842905 learned_addr learned my addr 172.21.15.72:0/1603842905 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:15.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f741f7fe640 1 -- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f742806c410 msgr2=0x7f742815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f742c8e0640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7428143e00 0x7f74280cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f7427fff640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f742806c410 0x7f742815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f741f7fe640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f742806c410 0x7f742815a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f741f7fe640 1 -- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7428143e00 msgr2=0x7f74280cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f741f7fe640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7428143e00 0x7f74280cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.140+0000 7f741f7fe640 1 -- 172.21.15.72:0/1603842905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74100094d0 con 0x7f7428072180 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.141+0000 7f742c8e0640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7428143e00 0x7f74280cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.141+0000 7f741f7fe640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f742813b4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7410000c00 tx=0x7f74100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.141+0000 7f74257fa640 1 -- 172.21.15.72:0/1603842905 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741003ac60 con 0x7f7428072180 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.141+0000 7f74257fa640 1 -- 172.21.15.72:0/1603842905 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741003adc0 con 0x7f7428072180 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.141+0000 7f74257fa640 1 -- 172.21.15.72:0/1603842905 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410044630 con 0x7f7428072180 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.141+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742813af90 con 0x7f7428072180 2023-12-12T09:52:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.141+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742815a6f0 con 0x7f7428072180 2023-12-12T09:52:15.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.142+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74280745d0 con 0x7f7428072180 2023-12-12T09:52:15.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.143+0000 7f74257fa640 1 -- 172.21.15.72:0/1603842905 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f741002f050 con 0x7f7428072180 2023-12-12T09:52:15.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.144+0000 7f74257fa640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f740806ae40 0x7f740806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.144+0000 7f74257fa640 1 -- 172.21.15.72:0/1603842905 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74100ab840 con 0x7f7428072180 2023-12-12T09:52:15.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.144+0000 7f7427fff640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f740806ae40 0x7f740806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.145+0000 7f7427fff640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f740806ae40 0x7f740806d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7418006fd0 tx=0x7f7418008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:15.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.148+0000 7f74257fa640 1 -- 172.21.15.72:0/1603842905 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7410076560 con 0x7f7428072180 2023-12-12T09:52:15.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.327+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f742813a830 con 0x7f7428072180 2023-12-12T09:52:15.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.329+0000 7f74257fa640 1 -- 172.21.15.72:0/1603842905 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f742813a830 con 0x7f7428072180 2023-12-12T09:52:15.331 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:15.332 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:15.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.331+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f740806ae40 msgr2=0x7f740806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.331+0000 7f742e343640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f740806ae40 0x7f740806d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7418006fd0 tx=0x7f7418008040 comp rx=0 tx=0).stop 2023-12-12T09:52:15.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.331+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 msgr2=0x7f742813b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.331+0000 7f742e343640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f742813b4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7410000c00 tx=0x7f74100046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.332+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 shutdown_connections 2023-12-12T09:52:15.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.332+0000 7f742e343640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f740806ae40 0x7f740806d300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.332+0000 7f742e343640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7428143e00 0x7f74280cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.332+0000 7f742e343640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7428072180 0x7f742813b4b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.332+0000 7f742e343640 1 --2- 172.21.15.72:0/1603842905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f742806c410 0x7f742815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.332+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 >> 172.21.15.72:0/1603842905 conn(0x7f74280b0a90 msgr2=0x7f742813d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:15.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.332+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 shutdown_connections 2023-12-12T09:52:15.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.332+0000 7f742e343640 1 -- 172.21.15.72:0/1603842905 wait complete. 2023-12-12T09:52:15.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-12T09:52:07.436771+0000', 'modified': '2023-12-12T09:52:12.519576+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4243}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4243': {'gid': 4243, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.72:6837/1645740011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6836', 'nonce': 1645740011}, {'type': 'v1', 'addr': '172.21.15.72:6837', 'nonce': 1645740011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-12-12T09:52:15.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-12T09:52:15.754 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-12T09:52:15.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.973+0000 7fb8cab39640 1 Processor -- start 2023-12-12T09:52:15.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.973+0000 7fb8cab39640 1 -- start start 2023-12-12T09:52:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.975+0000 7fb8cab39640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb8c415e2b0 0x7fb8c4164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.975+0000 7fb8cab39640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c406ca40 con 0x7fb8c4157a60 2023-12-12T09:52:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.975+0000 7fb8cab39640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c4071cd0 con 0x7fb8c406c410 2023-12-12T09:52:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.975+0000 7fb8cab39640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c4070f10 con 0x7fb8c415e2b0 2023-12-12T09:52:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.975+0000 7fb8c88d5640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fb8c4157a60 0x7fb8c406de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56630/0 (socket says 172.21.15.72:56630) 2023-12-12T09:52:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.975+0000 7fb8c88d5640 1 -- 172.21.15.72:0/3934221689 learned_addr learned my addr 172.21.15.72:0/3934221689 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.975+0000 7fb8c90d6640 1 --2- 172.21.15.72:0/3934221689 >> v2:172.21.15.72:3300/0 conn(0x7fb8c415e2b0 0x7fb8c4164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.976+0000 7fb8c37fe640 1 -- 172.21.15.72:0/3934221689 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542185522 0 0) 0x7fb8c406ca40 con 0x7fb8c4157a60 2023-12-12T09:52:15.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.976+0000 7fb8c37fe640 1 -- 172.21.15.72:0/3934221689 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b4003520 con 0x7fb8c4157a60 2023-12-12T09:52:15.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.976+0000 7fb8c90d6640 1 -- 172.21.15.72:0/3934221689 >> v1:172.21.15.113:6789/0 conn(0x7fb8c406c410 legacy=0x7fb8c415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:15.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.976+0000 7fb8c90d6640 1 -- 172.21.15.72:0/3934221689 >> v1:172.21.15.72:6789/0 conn(0x7fb8c4157a60 legacy=0x7fb8c406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.976+0000 7fb8c90d6640 1 -- 172.21.15.72:0/3934221689 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40711f0 con 0x7fb8c415e2b0 2023-12-12T09:52:15.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.977+0000 7fb8c90d6640 1 --2- 172.21.15.72:0/3934221689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c415e2b0 0x7fb8c4164d00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009d90 tx=0x7fb8bc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9b79bc883d5918 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:15.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.977+0000 7fb8c37fe640 1 -- 172.21.15.72:0/3934221689 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc02cbd0 con 0x7fb8c415e2b0 2023-12-12T09:52:15.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.977+0000 7fb8c37fe640 1 -- 172.21.15.72:0/3934221689 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc02cd30 con 0x7fb8c415e2b0 2023-12-12T09:52:15.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.977+0000 7fb8c37fe640 1 -- 172.21.15.72:0/3934221689 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc02cea0 con 0x7fb8c415e2b0 2023-12-12T09:52:15.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.978+0000 7fb8cab39640 1 -- 172.21.15.72:0/3934221689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c415e2b0 msgr2=0x7fb8c4164d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.978+0000 7fb8cab39640 1 --2- 172.21.15.72:0/3934221689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c415e2b0 0x7fb8c4164d00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009d90 tx=0x7fb8bc02be20 comp rx=0 tx=0).stop 2023-12-12T09:52:15.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.978+0000 7fb8cab39640 1 -- 172.21.15.72:0/3934221689 shutdown_connections 2023-12-12T09:52:15.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.978+0000 7fb8cab39640 1 --2- 172.21.15.72:0/3934221689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c415e2b0 0x7fb8c4164d00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.978+0000 7fb8cab39640 1 -- 172.21.15.72:0/3934221689 >> 172.21.15.72:0/3934221689 conn(0x7fb8c40b0a90 msgr2=0x7fb8c40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:15.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.978+0000 7fb8cab39640 1 -- 172.21.15.72:0/3934221689 shutdown_connections 2023-12-12T09:52:15.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.978+0000 7fb8cab39640 1 -- 172.21.15.72:0/3934221689 wait complete. 2023-12-12T09:52:15.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.980+0000 7fb8cab39640 1 Processor -- start 2023-12-12T09:52:15.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.980+0000 7fb8cab39640 1 -- start start 2023-12-12T09:52:15.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8cab39640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c406c410 0x7fb8c40b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8cab39640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4157a60 0x7fb8c40b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8cab39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c415e2b0 0x7fb8c41529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8cab39640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4155b90 con 0x7fb8c4157a60 2023-12-12T09:52:15.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8cab39640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4155e70 con 0x7fb8c406c410 2023-12-12T09:52:15.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8cab39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4156150 con 0x7fb8c415e2b0 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4157a60 0x7fb8c40b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4157a60 0x7fb8c40b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34246/0 (socket says 172.21.15.72:34246) 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c3fff640 1 -- 172.21.15.72:0/3514822661 learned_addr learned my addr 172.21.15.72:0/3514822661 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c406c410 0x7fb8c40b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c90d6640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c415e2b0 0x7fb8c41529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c3fff640 1 -- 172.21.15.72:0/3514822661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c415e2b0 msgr2=0x7fb8c41529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c415e2b0 0x7fb8c41529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c3fff640 1 -- 172.21.15.72:0/3514822661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c406c410 msgr2=0x7fb8c40b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c406c410 0x7fb8c40b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.982+0000 7fb8c3fff640 1 -- 172.21.15.72:0/3514822661 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc009a70 con 0x7fb8c4157a60 2023-12-12T09:52:15.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.983+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4157a60 0x7fb8c40b3430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c430 tx=0x7fb8b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:15.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.983+0000 7fb8c17fa640 1 -- 172.21.15.72:0/3514822661 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0011790 con 0x7fb8c4157a60 2023-12-12T09:52:15.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.983+0000 7fb8c17fa640 1 -- 172.21.15.72:0/3514822661 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b00118f0 con 0x7fb8c4157a60 2023-12-12T09:52:15.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.983+0000 7fb8c17fa640 1 -- 172.21.15.72:0/3514822661 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0011ce0 con 0x7fb8c4157a60 2023-12-12T09:52:15.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.983+0000 7fb8cab39640 1 -- 172.21.15.72:0/3514822661 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c4156430 con 0x7fb8c4157a60 2023-12-12T09:52:15.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.983+0000 7fb8cab39640 1 -- 172.21.15.72:0/3514822661 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c40b2d70 con 0x7fb8c4157a60 2023-12-12T09:52:15.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.985+0000 7fb8c17fa640 1 -- 172.21.15.72:0/3514822661 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb8b001c460 con 0x7fb8c4157a60 2023-12-12T09:52:15.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.986+0000 7fb8c17fa640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb8a806ae70 0x7fb8a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.986+0000 7fb8c17fa640 1 -- 172.21.15.72:0/3514822661 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb8b0089850 con 0x7fb8c4157a60 2023-12-12T09:52:15.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.986+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb8a806ae70 0x7fb8a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.987+0000 7fb8cd7a4740 1 Processor -- start 2023-12-12T09:52:15.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.987+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb8a806ae70 0x7fb8a806d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80011e0 tx=0x7fb8b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:15.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.987+0000 7fb8cd7a4740 1 -- start start 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463ed83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ea82c0 0x564463ed5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463eabb90 0x564463ed57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564463e1af40 con 0x564463ed8010 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564463e18630 con 0x564463eabb90 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564463e13f20 con 0x564463ea82c0 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8c88d5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463ed83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8c88d5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463ed83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34260/0 (socket says 172.21.15.72:34260) 2023-12-12T09:52:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.988+0000 7fb8c88d5640 1 -- 172.21.15.72:0/2878114829 learned_addr learned my addr 172.21.15.72:0/2878114829 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/2878114829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ea82c0 0x564463ed5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8c88d5640 1 -- 172.21.15.72:0/2878114829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ea82c0 msgr2=0x564463ed5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/2878114829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ea82c0 0x564463ed5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8c88d5640 1 -- 172.21.15.72:0/2878114829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463eabb90 msgr2=0x564463ed57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/2878114829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463eabb90 0x564463ed57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8c88d5640 1 -- 172.21.15.72:0/2878114829 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564463dd6910 con 0x564463ed8010 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/2878114829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463ed83f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8005880 tx=0x7fb8b8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd572de597e83da server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8ae7fc640 1 -- 172.21.15.72:0/2878114829 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8007710 con 0x564463ed8010 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8ae7fc640 1 -- 172.21.15.72:0/2878114829 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b800e7b0 con 0x564463ed8010 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.989+0000 7fb8ae7fc640 1 -- 172.21.15.72:0/2878114829 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800eb40 con 0x564463ed8010 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2878114829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 msgr2=0x564463ed83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2878114829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463ed83f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8005880 tx=0x7fb8b8011000 comp rx=0 tx=0).stop 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2878114829 shutdown_connections 2023-12-12T09:52:15.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2878114829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463eabb90 0x564463ed57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2878114829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ea82c0 0x564463ed5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2878114829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463ed83f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:15.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2878114829 >> 172.21.15.72:0/2878114829 conn(0x564463eaa530 msgr2=0x564463ef2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:15.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2878114829 shutdown_connections 2023-12-12T09:52:15.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.990+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2878114829 wait complete. 2023-12-12T09:52:15.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.991+0000 7fb8cd7a4740 1 -- start start 2023-12-12T09:52:15.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.991+0000 7fb8cd7a4740 1 Processor -- start 2023-12-12T09:52:15.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463ed9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ed8010 0x564463ee0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ee05b0 0x564463ec6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:15.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564463db02f0 con 0x564463ee05b0 2023-12-12T09:52:15.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564463dcdc70 con 0x564463ed8010 2023-12-12T09:52:16.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564463b76f20 con 0x564463eabb90 2023-12-12T09:52:16.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8cd7a4740 10 client.0 _ll_register_callbacks cb 0x564463dd23b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:52:16.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8c90d6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ee05b0 0x564463ec6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8c90d6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ee05b0 0x564463ec6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34270/0 (socket says 172.21.15.72:34270) 2023-12-12T09:52:16.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8c90d6640 1 -- 172.21.15.72:0/889126940 learned_addr learned my addr 172.21.15.72:0/889126940 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:16.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.992+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463ed9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ed8010 0x564463ee0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8c90d6640 1 -- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 msgr2=0x564463ed9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8c90d6640 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463ed9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8c90d6640 1 -- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ed8010 msgr2=0x564463ee0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8c90d6640 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ed8010 0x564463ee0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8c90d6640 1 -- 172.21.15.72:0/889126940 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b800f040 con 0x564463ee05b0 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ed8010 0x564463ee0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8c90d6640 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ee05b0 0x564463ec6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc037040 tx=0x7fb8bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b1cf1698611621 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8a37fe640 1 -- 172.21.15.72:0/889126940 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc02cea0 con 0x564463ee05b0 2023-12-12T09:52:16.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/889126940 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564463e0cdb0 con 0x564463ee05b0 2023-12-12T09:52:16.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.993+0000 7fb8a37fe640 1 -- 172.21.15.72:0/889126940 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb8bc034bd0 con 0x564463ee05b0 2023-12-12T09:52:16.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8a37fe640 1 -- 172.21.15.72:0/889126940 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8bc034d90 con 0x564463ee05b0 2023-12-12T09:52:16.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8cd7a4740 10 client.4386 fetch_fsmap learned FSMap version 6 2023-12-12T09:52:16.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/889126940 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x564463e125a0 con 0x564463ee05b0 2023-12-12T09:52:16.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8a37fe640 1 -- 172.21.15.72:0/889126940 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564463e0cdb0 con 0x564463ee05b0 2023-12-12T09:52:16.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8a37fe640 1 -- 172.21.15.72:0/889126940 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb8bc03f480 con 0x564463ee05b0 2023-12-12T09:52:16.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8cd7a4740 10 client.4386 fetch_fsmap finished waiting for FSMap version 6 2023-12-12T09:52:16.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8cd7a4740 10 client.4386 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8cd7a4740 20 client.4386 populate_metadata read hostname 'smithi072' 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x564463eddfc0 0x564463f06840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8cd7a4740 4 client.4386 mds_command: new command op to 4243 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/889126940 --> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564463e0cad0 con 0x564463eddfc0 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.994+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x564463eddfc0 0x564463f06840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.995+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x564463eddfc0 0x564463f06840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5e25b9f68669ac server_cookie=e482bb40f3a02052 in_seq=0 out_seq=0 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.995+0000 7fb8a37fe640 10 client.4386 ms_handle_connect on v2:172.21.15.72:6836/1645740011 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8a37fe640 1 -- 172.21.15.72:0/889126940 <== mds.0 v2:172.21.15.72:6836/1645740011 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564463e0cad0 con 0x564463eddfc0 2023-12-12T09:52:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8a37fe640 10 client.4386 handle_command_reply: tid=0 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 client.4386 shutdown 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/889126940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ee05b0 msgr2=0x564463ec6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ee05b0 0x564463ec6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc037040 tx=0x7fb8bc004a40 comp rx=0 tx=0).stop 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/889126940 shutdown_connections 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x564463eddfc0 0x564463f06840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ee05b0 0x564463ec6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ed8010 0x564463ee0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8a37fe640 0 client.4386 ms_handle_reset on v2:172.21.15.72:6836/1645740011 2023-12-12T09:52:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/889126940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463ed9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/889126940 >> 172.21.15.72:0/889126940 conn(0x564463ea82c0 msgr2=0x564463eac560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:16.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/889126940 shutdown_connections 2023-12-12T09:52:16.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/889126940 wait complete. 2023-12-12T09:52:16.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:15.996+0000 7fb8cd7a4740 20 client.4386 trim_cache size 0 max 16384 2023-12-12T09:52:16.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.013+0000 7fb8cd7a4740 1 Processor -- start 2023-12-12T09:52:16.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.013+0000 7fb8cd7a4740 1 -- start start 2023-12-12T09:52:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463eac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ee05b0 0x564463f014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564463ec9b10 con 0x564463ed8010 2023-12-12T09:52:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564463e46010 con 0x564463ee05b0 2023-12-12T09:52:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564463ef3af0 con 0x564463eabb90 2023-12-12T09:52:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34274/0 (socket says 172.21.15.72:34274) 2023-12-12T09:52:16.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.014+0000 7fb8c3fff640 1 -- 172.21.15.72:0/3150360799 learned_addr learned my addr 172.21.15.72:0/3150360799 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:16.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c90d6640 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ee05b0 0x564463f014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463eac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c3fff640 1 -- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 msgr2=0x564463eac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463eac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c3fff640 1 -- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ee05b0 msgr2=0x564463f014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ee05b0 0x564463f014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c3fff640 1 -- 172.21.15.72:0/3150360799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564463dafb80 con 0x564463ed8010 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463eac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000cdf0 tx=0x7fb8b0089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646b2c485ea5540f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8aeffd640 1 -- 172.21.15.72:0/3150360799 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b005c410 con 0x564463ed8010 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8aeffd640 1 -- 172.21.15.72:0/3150360799 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b00605c0 con 0x564463ed8010 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.015+0000 7fb8aeffd640 1 -- 172.21.15.72:0/3150360799 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0056660 con 0x564463ed8010 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/3150360799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 msgr2=0x564463eac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000cdf0 tx=0x7fb8b0089040 comp rx=0 tx=0).stop 2023-12-12T09:52:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/3150360799 shutdown_connections 2023-12-12T09:52:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463ee05b0 0x564463f014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/3150360799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463eabb90 0x564463eac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/3150360799 >> 172.21.15.72:0/3150360799 conn(0x564463ea82c0 msgr2=0x564463ea3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/3150360799 shutdown_connections 2023-12-12T09:52:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.016+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/3150360799 wait complete. 2023-12-12T09:52:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.017+0000 7fb8cd7a4740 1 -- start start 2023-12-12T09:52:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.017+0000 7fb8cd7a4740 1 Processor -- start 2023-12-12T09:52:16.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463eabb90 0x564463eac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8cd7a4740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ee05b0 0x564463f0c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564463ef24e0 con 0x564463ed8010 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564463f165c0 con 0x564463eabb90 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8cd7a4740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564463f47220 con 0x564463ee05b0 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8cd7a4740 10 client.0 _ll_register_callbacks cb 0x7fb8bc037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34284/0 (socket says 172.21.15.72:34284) 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8c3fff640 1 -- 172.21.15.72:0/2201955388 learned_addr learned my addr 172.21.15.72:0/2201955388 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8c3fff640 1 -- 172.21.15.72:0/2201955388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ee05b0 msgr2=0x564463f0c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ee05b0 0x564463f0c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8c3fff640 1 -- 172.21.15.72:0/2201955388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463eabb90 msgr2=0x564463eac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463eabb90 0x564463eac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.018+0000 7fb8c3fff640 1 -- 172.21.15.72:0/2201955388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c40b2d70 con 0x564463ed8010 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8c3fff640 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000cdf0 tx=0x7fb8b001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2ddd8a27a1b22f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8a37fe640 1 -- 172.21.15.72:0/2201955388 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b005c410 con 0x564463ed8010 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8a37fe640 1 -- 172.21.15.72:0/2201955388 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb8b00549d0 con 0x564463ed8010 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2201955388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8b800e7b0 con 0x564463ed8010 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8a37fe640 1 -- 172.21.15.72:0/2201955388 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8b0054bb0 con 0x564463ed8010 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8cd7a4740 10 client.4392 fetch_fsmap learned FSMap version 6 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2201955388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x564463dd5170 con 0x564463ed8010 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8a37fe640 1 -- 172.21.15.72:0/2201955388 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b800e7b0 con 0x564463ed8010 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8a37fe640 1 -- 172.21.15.72:0/2201955388 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb8b0060130 con 0x564463ed8010 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8cd7a4740 10 client.4392 fetch_fsmap finished waiting for FSMap version 6 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8cd7a4740 10 client.4392 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8cd7a4740 20 client.4392 populate_metadata read hostname 'smithi072' 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.019+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x564463f0ce30 0x564463f5df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.020+0000 7fb8cd7a4740 4 client.4392 mds_command: new command op to 4243 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.020+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2201955388 --> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564463b146c0 con 0x564463f0ce30 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.020+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x564463f0ce30 0x564463f5df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.020+0000 7fb8c88d5640 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x564463f0ce30 0x564463f5df60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967f0f05056c4c5c server_cookie=44b390b13ef3b33b in_seq=0 out_seq=0 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.020+0000 7fb8a37fe640 10 client.4392 ms_handle_connect on v2:172.21.15.72:6836/1645740011 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.020+0000 7fb8a37fe640 1 -- 172.21.15.72:0/2201955388 <== mds.0 v2:172.21.15.72:6836/1645740011 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x564463b146c0 con 0x564463f0ce30 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.020+0000 7fb8a37fe640 10 client.4392 handle_command_reply: tid=0 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 client.4392 shutdown 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2201955388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 msgr2=0x564463eac5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000cdf0 tx=0x7fb8b001c000 comp rx=0 tx=0).stop 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2201955388 shutdown_connections 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x564463f0ce30 0x564463f5df60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564463ee05b0 0x564463f0c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564463ed8010 0x564463eac5a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 --2- 172.21.15.72:0/2201955388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564463eabb90 0x564463eac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2201955388 >> 172.21.15.72:0/2201955388 conn(0x564463ea82c0 msgr2=0x564463ea72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:16.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8a37fe640 0 client.4392 ms_handle_reset on v2:172.21.15.72:6836/1645740011 2023-12-12T09:52:16.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2201955388 shutdown_connections 2023-12-12T09:52:16.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 1 -- 172.21.15.72:0/2201955388 wait complete. 2023-12-12T09:52:16.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.021+0000 7fb8cd7a4740 20 client.4392 trim_cache size 0 max 16384 2023-12-12T09:52:16.027 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:52:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "bd07f350-42f2-4d45-9a3d-13364a8566fe", 2023-12-12T09:52:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 4243, 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 8.0015624600000006, 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 6, 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 24, 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 24, 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 15.234954909000001 2023-12-12T09:52:16.028 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:52:16.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 -- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb8a806ae70 msgr2=0x7fb8a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb8a806ae70 0x7fb8a806d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80011e0 tx=0x7fb8b8009040 comp rx=0 tx=0).stop 2023-12-12T09:52:16.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 -- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4157a60 msgr2=0x7fb8c40b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4157a60 0x7fb8c40b3430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c430 tx=0x7fb8b000c8f0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 -- 172.21.15.72:0/3514822661 shutdown_connections 2023-12-12T09:52:16.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb8a806ae70 0x7fb8a806d330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c415e2b0 0x7fb8c41529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4157a60 0x7fb8c40b3430 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 --2- 172.21.15.72:0/3514822661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c406c410 0x7fb8c40b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 -- 172.21.15.72:0/3514822661 >> 172.21.15.72:0/3514822661 conn(0x7fb8c40b0a90 msgr2=0x7fb8c415a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:16.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 -- 172.21.15.72:0/3514822661 shutdown_connections 2023-12-12T09:52:16.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.022+0000 7fb8cab39640 1 -- 172.21.15.72:0/3514822661 wait complete. 2023-12-12T09:52:16.043 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-12T09:52:16.043 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:16.043 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-12T09:52:16.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.238+0000 7f96d9943640 1 Processor -- start 2023-12-12T09:52:16.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.238+0000 7f96d9943640 1 -- start start 2023-12-12T09:52:16.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.240+0000 7f96d9943640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f96d415e2b0 0x7f96d4164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.240+0000 7f96d9943640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d406ca40 con 0x7f96d406c410 2023-12-12T09:52:16.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.240+0000 7f96d9943640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d4071cd0 con 0x7f96d4157a60 2023-12-12T09:52:16.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.240+0000 7f96d9943640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d4070f10 con 0x7f96d415e2b0 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96cbfff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f96d406c410 0x7f96d415dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56632/0 (socket says 172.21.15.72:56632) 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96cbfff640 1 -- 172.21.15.72:0/3517375678 learned_addr learned my addr 172.21.15.72:0/3517375678 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d37fe640 1 --2- 172.21.15.72:0/3517375678 >> v2:172.21.15.113:3300/0 conn(0x7f96d415e2b0 0x7f96d4164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407142866 0 0) 0x7f96d406ca40 con 0x7f96d406c410 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c0003520 con 0x7f96d406c410 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3505374616 0 0) 0x7f96c0003520 con 0x7f96d406c410 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 >> v2:172.21.15.113:3300/0 conn(0x7f96d415e2b0 msgr2=0x7f96d4164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 --2- 172.21.15.72:0/3517375678 >> v2:172.21.15.113:3300/0 conn(0x7f96d415e2b0 0x7f96d4164d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 >> v1:172.21.15.113:6789/0 conn(0x7f96d4157a60 legacy=0x7f96d406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d40711f0 con 0x7f96d406c410 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f96bc001670 con 0x7f96d406c410 2023-12-12T09:52:16.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 >> v1:172.21.15.72:6789/0 conn(0x7f96d406c410 legacy=0x7f96d415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.241+0000 7f96d27fc640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c00039c0 0x7f96c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d27fc640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c00047c0 0x7f96c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d27fc640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96c000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d4070f10 con 0x7f96c00051d0 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c000e330 con 0x7f96c00039c0 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c000e4f0 con 0x7f96c00047c0 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d37fe640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96c000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d2ffd640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c00039c0 0x7f96c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d37fe640 1 -- 172.21.15.72:0/3517375678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c00047c0 msgr2=0x7f96c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d37fe640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c00047c0 0x7f96c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d37fe640 1 -- 172.21.15.72:0/3517375678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c00039c0 msgr2=0x7f96c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d37fe640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c00039c0 0x7f96c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.242+0000 7f96d37fe640 1 -- 172.21.15.72:0/3517375678 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c0003520 con 0x7f96c00051d0 2023-12-12T09:52:16.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.243+0000 7f96d37fe640 1 -- 172.21.15.72:0/3517375678 reap_dead start 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.243+0000 7f96d37fe640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96c000db90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009f90 tx=0x7f96c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c0a5e63c9649c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.243+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc02cbf0 con 0x7f96c00051d0 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.243+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc00f040 con 0x7f96c00051d0 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.243+0000 7f96d27fc640 1 -- 172.21.15.72:0/3517375678 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc003990 con 0x7f96c00051d0 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.244+0000 7f96d9943640 1 -- 172.21.15.72:0/3517375678 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 msgr2=0x7f96c000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.244+0000 7f96d9943640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96c000db90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009f90 tx=0x7f96c00057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.244+0000 7f96d9943640 1 -- 172.21.15.72:0/3517375678 shutdown_connections 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.244+0000 7f96d9943640 1 --2- 172.21.15.72:0/3517375678 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96c000db90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.244+0000 7f96d9943640 1 -- 172.21.15.72:0/3517375678 >> 172.21.15.72:0/3517375678 conn(0x7f96d40b0a90 msgr2=0x7f96d40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.244+0000 7f96d9943640 1 -- 172.21.15.72:0/3517375678 shutdown_connections 2023-12-12T09:52:16.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.244+0000 7f96d9943640 1 -- 172.21.15.72:0/3517375678 wait complete. 2023-12-12T09:52:16.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.246+0000 7f96d9943640 1 Processor -- start 2023-12-12T09:52:16.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.246+0000 7f96d9943640 1 -- start start 2023-12-12T09:52:16.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d9943640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96d41476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d9943640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96d40b4af0 0x7f96d40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d9943640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96d40cce10 0x7f96d415b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d9943640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40cd290 con 0x7f96c00051d0 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d9943640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d41524c0 con 0x7f96d40cce10 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d9943640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d4147be0 con 0x7f96d40b4af0 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96d41476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96d41476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34290/0 (socket says 172.21.15.72:34290) 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.247+0000 7f96d2ffd640 1 -- 172.21.15.72:0/2982639141 learned_addr learned my addr 172.21.15.72:0/2982639141 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d37fe640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96d40cce10 0x7f96d415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96cbfff640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96d40b4af0 0x7f96d40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d2ffd640 1 -- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96d40b4af0 msgr2=0x7f96d40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d2ffd640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96d40b4af0 0x7f96d40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d2ffd640 1 -- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96d40cce10 msgr2=0x7f96d415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d2ffd640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96d40cce10 0x7f96d415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d2ffd640 1 -- 172.21.15.72:0/2982639141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d414f740 con 0x7f96c00051d0 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d37fe640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96d40cce10 0x7f96d415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:16.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96cbfff640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96d40b4af0 0x7f96d40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:16.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d2ffd640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96d41476a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96c400e4d0 tx=0x7f96c400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:16.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.248+0000 7f96d0ff9640 1 -- 172.21.15.72:0/2982639141 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40052e0 con 0x7f96c00051d0 2023-12-12T09:52:16.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.249+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d414fa20 con 0x7f96c00051d0 2023-12-12T09:52:16.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.249+0000 7f96d0ff9640 1 -- 172.21.15.72:0/2982639141 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c4005440 con 0x7f96c00051d0 2023-12-12T09:52:16.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.249+0000 7f96d0ff9640 1 -- 172.21.15.72:0/2982639141 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4003810 con 0x7f96c00051d0 2023-12-12T09:52:16.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.249+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d4152150 con 0x7f96c00051d0 2023-12-12T09:52:16.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.250+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d40745d0 con 0x7f96c00051d0 2023-12-12T09:52:16.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.251+0000 7f96d0ff9640 1 -- 172.21.15.72:0/2982639141 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f96c4003970 con 0x7f96c00051d0 2023-12-12T09:52:16.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.251+0000 7f96d0ff9640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f96b406ae70 0x7f96b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:16.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.252+0000 7f96d0ff9640 1 -- 172.21.15.72:0/2982639141 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f96c40896b0 con 0x7f96c00051d0 2023-12-12T09:52:16.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.252+0000 7f96cbfff640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f96b406ae70 0x7f96b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:16.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.252+0000 7f96cbfff640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f96b406ae70 0x7f96b406d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96d4156e50 tx=0x7f96bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:16.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.256+0000 7f96d0ff9640 1 -- 172.21.15.72:0/2982639141 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96c40543f0 con 0x7f96c00051d0 2023-12-12T09:52:16.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.487+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96d415de50 con 0x7f96c00051d0 2023-12-12T09:52:16.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.488+0000 7f96d0ff9640 1 -- 172.21.15.72:0/2982639141 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f96c4057a10 con 0x7f96c00051d0 2023-12-12T09:52:16.489 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-12T09:52:16.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f96b406ae70 msgr2=0x7f96b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f96b406ae70 0x7f96b406d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96d4156e50 tx=0x7f96bc009040 comp rx=0 tx=0).stop 2023-12-12T09:52:16.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 msgr2=0x7f96d41476a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96d41476a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96c400e4d0 tx=0x7f96c400e990 comp rx=0 tx=0).stop 2023-12-12T09:52:16.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 shutdown_connections 2023-12-12T09:52:16.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f96b406ae70 0x7f96b406d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96d40cce10 0x7f96d415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96d40b4af0 0x7f96d40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 --2- 172.21.15.72:0/2982639141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f96c00051d0 0x7f96d41476a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 >> 172.21.15.72:0/2982639141 conn(0x7f96d40b0a90 msgr2=0x7f96d406dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:16.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 shutdown_connections 2023-12-12T09:52:16.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:16.490+0000 7f96d9943640 1 -- 172.21.15.72:0/2982639141 wait complete. 2023-12-12T09:52:16.509 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-12T09:52:16.509 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:52:16.510 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2023-12-12T09:52:16.510 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-12-12T09:52:16.510 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:16.510 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-12T09:52:16.510 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-12T09:52:16.510 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:52:16.510 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-12-12T09:52:16.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:16.510 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2023-12-12T09:52:16.558 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:52:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:52:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-12T09:52:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 42366sec preferred_lft 42366sec 2023-12-12T09:52:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2023-12-12T09:52:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:16.560 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-12T09:52:16.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:16.561 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:16.561 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:16.561 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add name ceph-brx type bridge 2023-12-12T09:52:16.561 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr flush dev ceph-brx 2023-12-12T09:52:16.561 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set ceph-brx up 2023-12-12T09:52:16.561 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-12T09:52:16.561 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:17.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:17.134 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:52:17.220 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-12-12T09:52:17.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:17.223 DEBUG:teuthology.orchestra.run.smithi072:> route 2023-12-12T09:52:17.305 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2023-12-12T09:52:17.306 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:52:17.306 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-12T09:52:17.306 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-12T09:52:17.306 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:52:17.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:17.307 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:17.307 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:17.307 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-12T09:52:17.307 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-12T09:52:17.307 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-12T09:52:17.307 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:18.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:18.345 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:52:18.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:18.398 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2023-12-12T09:52:18.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:18.445 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:18.445 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:18.445 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:52:18.445 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-12-12T09:52:18.445 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:18.636 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-12-12T09:52:18.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:18.637 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:18.637 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:18.637 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-12-12T09:52:18.637 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-12T09:52:18.637 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-12-12T09:52:18.637 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-12-12T09:52:18.637 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-12-12T09:52:18.637 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:18.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:18.921 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:18.921 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:18.921 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2023-12-12T09:52:18.921 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.0 master ceph-brx 2023-12-12T09:52:18.921 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:19.084 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-12-12T09:52:19.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:52:19.084 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:19.135 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-12-12T09:52:19.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:52:19.136 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:19.230 DEBUG:teuthology.orchestra.run.smithi072:> sudo modprobe fuse 2023-12-12T09:52:19.308 DEBUG:teuthology.orchestra.run.smithi072:> cat /proc/self/mounts | awk '{print $2}' 2023-12-12T09:52:19.355 INFO:teuthology.orchestra.run.smithi072.stdout:/proc 2023-12-12T09:52:19.355 INFO:teuthology.orchestra.run.smithi072.stdout:/sys 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/dev 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/security 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/shm 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/pts 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/run 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/cgroup 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/pstore 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/ 2023-12-12T09:52:19.356 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/selinux 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/hugepages 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/mqueue 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/debug 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/tracing 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/fuse/connections 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/config 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-sysctl.service 2023-12-12T09:52:19.357 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/nfs/rpc_pipefs 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/1000 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-0 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-1 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-2 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-3 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:52:19.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:19.359 DEBUG:teuthology.orchestra.run.smithi072:> ls /sys/fs/fuse/connections 2023-12-12T09:52:19.452 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-12T09:52:19.453 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-12-12T09:52:19.458 DEBUG:teuthology.orchestra.run.smithi072:> sudo modprobe fuse 2023-12-12T09:52:19.534 DEBUG:teuthology.orchestra.run.smithi072:> cat /proc/self/mounts | awk '{print $2}' 2023-12-12T09:52:19.575 INFO:teuthology.orchestra.run.smithi072.stdout:/proc 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/sys 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/dev 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/security 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/shm 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/pts 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/run 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/cgroup 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/pstore 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/ 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/selinux 2023-12-12T09:52:19.576 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/hugepages 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/mqueue 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/debug 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/tracing 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/fuse/connections 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/config 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-sysctl.service 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/nfs/rpc_pipefs 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/1000 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-0 2023-12-12T09:52:19.577 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-1 2023-12-12T09:52:19.578 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-2 2023-12-12T09:52:19.578 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-3 2023-12-12T09:52:19.578 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns 2023-12-12T09:52:19.578 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:52:19.578 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:52:19.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:19.579 DEBUG:teuthology.orchestra.run.smithi072:> ls /sys/fs/fuse/connections 2023-12-12T09:52:19.663 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi072.stderr:2023-12-12T09:52:19.661+0000 7fa076241480 -1 init, newargv = 0x55f8700b65e0 newargc=15 2023-12-12T09:52:19.663 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi072.stderr:ceph-fuse[95124]: starting ceph client 2023-12-12T09:52:19.674 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi072.stderr:ceph-fuse[95124]: starting fuse 2023-12-12T09:52:20.770 DEBUG:teuthology.orchestra.run.smithi072:> sudo modprobe fuse 2023-12-12T09:52:20.873 DEBUG:teuthology.orchestra.run.smithi072:> cat /proc/self/mounts | awk '{print $2}' 2023-12-12T09:52:20.932 INFO:teuthology.orchestra.run.smithi072.stdout:/proc 2023-12-12T09:52:20.932 INFO:teuthology.orchestra.run.smithi072.stdout:/sys 2023-12-12T09:52:20.932 INFO:teuthology.orchestra.run.smithi072.stdout:/dev 2023-12-12T09:52:20.932 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/security 2023-12-12T09:52:20.932 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/shm 2023-12-12T09:52:20.932 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/pts 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/run 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/cgroup 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/pstore 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/ 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/selinux 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/hugepages 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/mqueue 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/debug 2023-12-12T09:52:20.933 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/tracing 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/fuse/connections 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/config 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-sysctl.service 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/nfs/rpc_pipefs 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/1000 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-0 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-1 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-2 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph/osd/ceph-3 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns 2023-12-12T09:52:20.934 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:52:20.935 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:52:20.935 INFO:teuthology.orchestra.run.smithi072.stdout:/home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:20.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:20.936 DEBUG:teuthology.orchestra.run.smithi072:> ls /sys/fs/fuse/connections 2023-12-12T09:52:20.987 INFO:teuthology.orchestra.run.smithi072.stdout:39 2023-12-12T09:52:20.987 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-12-12T09:52:20.987 DEBUG:teuthology.orchestra.run.smithi072:> sudo stdin-killer -- python3 -c ' 2023-12-12T09:52:20.987 DEBUG:teuthology.orchestra.run.smithi072:> import glob 2023-12-12T09:52:20.987 DEBUG:teuthology.orchestra.run.smithi072:> import re 2023-12-12T09:52:20.987 DEBUG:teuthology.orchestra.run.smithi072:> import os 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> import subprocess 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> def _find_admin_socket(client_name): 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> files = glob.glob(asok_path) 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> # Given a non-glob path, it better be there 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> if "*" not in asok_path: 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> assert(len(files) == 1) 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> return files[0] 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> for f in files: 2023-12-12T09:52:20.988 DEBUG:teuthology.orchestra.run.smithi072:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> contents = proc_f.read() 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> if mountpoint in contents: 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> return f 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> print(_find_admin_socket("client.admin")) 2023-12-12T09:52:20.989 DEBUG:teuthology.orchestra.run.smithi072:> ' 2023-12-12T09:52:21.165 INFO:teuthology.orchestra.run.smithi072.stdout:/var/run/ceph/ceph-client.admin.95124.asok 2023-12-12T09:52:21.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-12T09:52:21.182 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.95124.asok 2023-12-12T09:52:21.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:21.183 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.95124.asok status 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: "metadata": { 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_sha1": "f1fbf3c583d03e19f5e58eb618e78d8e10d075f9", 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev)", 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: "entity_id": "admin", 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": "smithi072", 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: "pid": "95124", 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: "root": "/" 2023-12-12T09:52:21.424 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "dentry_count": 0, 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "dentry_pinned_count": 0, 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 4404, 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "inst": { 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "name": { 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "client", 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "num": 4404 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": { 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "192.168.0.1:0", 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 2519460324 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:52:21.425 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": { 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "192.168.0.1:0", 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 2519460324 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "inst_str": "client.4404 192.168.0.1:0/2519460324", 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "addr_str": "192.168.0.1:0/2519460324", 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "inode_count": 1, 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_epoch": 6, 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_epoch": 24, 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_epoch_barrier": 0, 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "blocklisted": false, 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs" 2023-12-12T09:52:21.426 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:52:21.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:21.439 DEBUG:teuthology.orchestra.run.smithi072:> stat --file-system '--printf=%T 2023-12-12T09:52:21.439 DEBUG:teuthology.orchestra.run.smithi072:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:21.494 INFO:teuthology.orchestra.run.smithi072.stdout:fuseblk 2023-12-12T09:52:21.494 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:21.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:21.494 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:21.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:21.591 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-12-12T09:52:21.690 INFO:teuthology.orchestra.run.smithi072.stdout:# file: . 2023-12-12T09:52:21.691 INFO:teuthology.orchestra.run.smithi072.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-12T09:52:21.691 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:21.692 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:52:21.692 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:52:21.746 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.958+0000 7f0c931e3640 1 Processor -- start 2023-12-12T09:52:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.958+0000 7f0c931e3640 1 -- start start 2023-12-12T09:52:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.960+0000 7f0c931e3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c8c143e00 0x7f0c8c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.960+0000 7f0c931e3640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8c0b3bb0 con 0x7f0c8c06c410 2023-12-12T09:52:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.960+0000 7f0c931e3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8c1441e0 con 0x7f0c8c072180 2023-12-12T09:52:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.960+0000 7f0c931e3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c072680 con 0x7f0c8c143e00 2023-12-12T09:52:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.960+0000 7f0c91780640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f0c8c06c410 0x7f0c8c15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56646/0 (socket says 172.21.15.72:56646) 2023-12-12T09:52:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.960+0000 7f0c91780640 1 -- 172.21.15.72:0/3492580521 learned_addr learned my addr 172.21.15.72:0/3492580521 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c90f7f640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c8c143e00 0x7f0c8c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237778039 0 0) 0x7f0c8c0b3bb0 con 0x7f0c8c06c410 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c7c003520 con 0x7f0c8c06c410 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2299388795 0 0) 0x7f0c7c003520 con 0x7f0c8c06c410 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c8c143e00 msgr2=0x7f0c8c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c8c143e00 0x7f0c8c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 >> v1:172.21.15.113:6789/0 conn(0x7f0c8c072180 legacy=0x7f0c8c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c0b1e30 con 0x7f0c8c06c410 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f0c840019f0 con 0x7f0c8c06c410 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 >> v1:172.21.15.72:6789/0 conn(0x7f0c8c06c410 legacy=0x7f0c8c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7c0039a0 0x7f0c7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c8c143e00 0x7f0c7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c7c004d00 0x7f0c7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c072680 con 0x7f0c7c004d00 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c00ba30 con 0x7f0c7c0039a0 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.961+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c00bbf0 con 0x7f0c8c143e00 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.962+0000 7f0c91780640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c7c004d00 0x7f0c7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.962+0000 7f0c90f7f640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7c0039a0 0x7f0c7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.962+0000 7f0c8bfff640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c8c143e00 0x7f0c7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.962+0000 7f0c91780640 1 -- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c8c143e00 msgr2=0x7f0c7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.962+0000 7f0c91780640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c8c143e00 0x7f0c7c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.962+0000 7f0c91780640 1 -- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7c0039a0 msgr2=0x7f0c7c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.962+0000 7f0c91780640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c7c0039a0 0x7f0c7c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.962+0000 7f0c91780640 1 -- 172.21.15.72:0/3492580521 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c003520 con 0x7f0c7c004d00 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.963+0000 7f0c91780640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c7c004d00 0x7f0c7c005180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400a800 tx=0x7f0c7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dfac5173d849c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.963+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c840072d0 con 0x7f0c7c004d00 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.963+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8402f070 con 0x7f0c7c004d00 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.963+0000 7f0c8b7fe640 1 -- 172.21.15.72:0/3492580521 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84007690 con 0x7f0c7c004d00 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.964+0000 7f0c931e3640 1 -- 172.21.15.72:0/3492580521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c7c004d00 msgr2=0x7f0c7c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.964+0000 7f0c931e3640 1 --2- 172.21.15.72:0/3492580521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c7c004d00 0x7f0c7c005180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400a800 tx=0x7f0c7c0057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.964+0000 7f0c91780640 1 -- 172.21.15.72:0/3492580521 reap_dead start 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.964+0000 7f0c931e3640 1 -- 172.21.15.72:0/3492580521 shutdown_connections 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.964+0000 7f0c931e3640 1 -- 172.21.15.72:0/3492580521 >> 172.21.15.72:0/3492580521 conn(0x7f0c8c0b0a90 msgr2=0x7f0c8c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.964+0000 7f0c931e3640 1 -- 172.21.15.72:0/3492580521 shutdown_connections 2023-12-12T09:52:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.964+0000 7f0c931e3640 1 -- 172.21.15.72:0/3492580521 wait complete. 2023-12-12T09:52:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.966+0000 7f0c931e3640 1 Processor -- start 2023-12-12T09:52:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.966+0000 7f0c931e3640 1 -- start start 2023-12-12T09:52:21.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c931e3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7c004d00 0x7f0c8c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c931e3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c8c13a190 0x7f0c8c15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c931e3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c8c13d220 0x7f0c8c13d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c931e3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c13dbe0 con 0x7f0c8c13d220 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c931e3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c15ad60 con 0x7f0c8c13a190 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c931e3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c0cd6b0 con 0x7f0c7c004d00 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c91780640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c8c13d220 0x7f0c8c13d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c91780640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c8c13d220 0x7f0c8c13d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34306/0 (socket says 172.21.15.72:34306) 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c91780640 1 -- 172.21.15.72:0/1803630130 learned_addr learned my addr 172.21.15.72:0/1803630130 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.967+0000 7f0c8bfff640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c8c13a190 0x7f0c8c15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c90f7f640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7c004d00 0x7f0c8c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c91780640 1 -- 172.21.15.72:0/1803630130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7c004d00 msgr2=0x7f0c8c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:21.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c91780640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7c004d00 0x7f0c8c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c91780640 1 -- 172.21.15.72:0/1803630130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c8c13a190 msgr2=0x7f0c8c15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:21.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c91780640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c8c13a190 0x7f0c8c15a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c91780640 1 -- 172.21.15.72:0/1803630130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c1f6cc0 con 0x7f0c8c13d220 2023-12-12T09:52:21.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c91780640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c8c13d220 0x7f0c8c13d6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c8402cb50 tx=0x7f0c84001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:21.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c897fa640 1 -- 172.21.15.72:0/1803630130 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84004d60 con 0x7f0c8c13d220 2023-12-12T09:52:21.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c897fa640 1 -- 172.21.15.72:0/1803630130 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8402ce90 con 0x7f0c8c13d220 2023-12-12T09:52:21.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c897fa640 1 -- 172.21.15.72:0/1803630130 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84007310 con 0x7f0c8c13d220 2023-12-12T09:52:21.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.968+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c15b050 con 0x7f0c8c13d220 2023-12-12T09:52:21.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.969+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c0cc8f0 con 0x7f0c8c13d220 2023-12-12T09:52:21.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.970+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c0745d0 con 0x7f0c8c13d220 2023-12-12T09:52:21.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.971+0000 7f0c897fa640 1 -- 172.21.15.72:0/1803630130 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c84031030 con 0x7f0c8c13d220 2023-12-12T09:52:21.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.971+0000 7f0c897fa640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0c7006ae40 0x7f0c7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.971+0000 7f0c897fa640 1 -- 172.21.15.72:0/1803630130 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0c840ab460 con 0x7f0c8c13d220 2023-12-12T09:52:21.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.972+0000 7f0c90f7f640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0c7006ae40 0x7f0c7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.972+0000 7f0c90f7f640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0c7006ae40 0x7f0c7006d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009a10 tx=0x7f0c80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:21.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:21.976+0000 7f0c897fa640 1 -- 172.21.15.72:0/1803630130 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c84076210 con 0x7f0c8c13d220 2023-12-12T09:52:22.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.168+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c8c0745d0 con 0x7f0c8c13d220 2023-12-12T09:52:22.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.171+0000 7f0c897fa640 1 -- 172.21.15.72:0/1803630130 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f0c84079830 con 0x7f0c8c13d220 2023-12-12T09:52:22.173 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:22.173 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:22.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0c7006ae40 msgr2=0x7f0c7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0c7006ae40 0x7f0c7006d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009a10 tx=0x7f0c80008040 comp rx=0 tx=0).stop 2023-12-12T09:52:22.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c8c13d220 msgr2=0x7f0c8c13d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c8c13d220 0x7f0c8c13d6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c8402cb50 tx=0x7f0c84001d80 comp rx=0 tx=0).stop 2023-12-12T09:52:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 shutdown_connections 2023-12-12T09:52:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0c7006ae40 0x7f0c7006d300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c8c13d220 0x7f0c8c13d6a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c8c13a190 0x7f0c8c15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 --2- 172.21.15.72:0/1803630130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c7c004d00 0x7f0c8c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.173+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 >> 172.21.15.72:0/1803630130 conn(0x7f0c8c0b0a90 msgr2=0x7f0c8c0afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.174+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 shutdown_connections 2023-12-12T09:52:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.174+0000 7f0c931e3640 1 -- 172.21.15.72:0/1803630130 wait complete. 2023-12-12T09:52:22.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:52:22.194 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:52:22.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.407+0000 7f75903c9640 1 Processor -- start 2023-12-12T09:52:22.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.407+0000 7f75903c9640 1 -- start start 2023-12-12T09:52:22.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.408+0000 7f75903c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 0x7f758806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.408+0000 7f75903c9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75880ce7e0 0x7f758815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.409+0000 7f75903c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7588071cd0 con 0x7f758806a460 2023-12-12T09:52:22.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.409+0000 7f75903c9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75880717e0 con 0x7f75880ce7e0 2023-12-12T09:52:22.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.409+0000 7f75903c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758806c6b0 con 0x7f758815af50 2023-12-12T09:52:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.409+0000 7f758d964640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75880ce7e0 0x7f758815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.409+0000 7f758d964640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75880ce7e0 0x7f758815ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:56328/0 (socket says 172.21.15.72:56328) 2023-12-12T09:52:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.409+0000 7f758e165640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 0x7f758806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.409+0000 7f758d964640 1 -- 172.21.15.72:0/1935156338 learned_addr learned my addr 172.21.15.72:0/1935156338 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:22.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.410+0000 7f758d964640 1 -- 172.21.15.72:0/1935156338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 msgr2=0x7f758806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:22.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.410+0000 7f758d964640 1 --2- 172.21.15.72:0/1935156338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 0x7f758806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.410+0000 7f758d964640 1 -- 172.21.15.72:0/1935156338 >> v1:172.21.15.113:6789/0 conn(0x7f758806a460 legacy=0x7f75880ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.410+0000 7f758d964640 1 -- 172.21.15.72:0/1935156338 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880719f0 con 0x7f75880ce7e0 2023-12-12T09:52:22.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.411+0000 7f758d964640 1 --2- 172.21.15.72:0/1935156338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 0x7f758815ec00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f757c009870 tx=0x7f757c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d59738f21feea46 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:22.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f758d163640 1 -- 172.21.15.72:0/1935156338 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c0317c0 con 0x7f75880ce7e0 2023-12-12T09:52:22.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f758d163640 1 -- 172.21.15.72:0/1935156338 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c031920 con 0x7f75880ce7e0 2023-12-12T09:52:22.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f758d163640 1 -- 172.21.15.72:0/1935156338 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c031cb0 con 0x7f75880ce7e0 2023-12-12T09:52:22.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f75903c9640 1 -- 172.21.15.72:0/1935156338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 msgr2=0x7f758815ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f75903c9640 1 --2- 172.21.15.72:0/1935156338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 0x7f758815ec00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f757c009870 tx=0x7f757c02b860 comp rx=0 tx=0).stop 2023-12-12T09:52:22.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f75903c9640 1 -- 172.21.15.72:0/1935156338 shutdown_connections 2023-12-12T09:52:22.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f75903c9640 1 --2- 172.21.15.72:0/1935156338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 0x7f758815ec00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f75903c9640 1 --2- 172.21.15.72:0/1935156338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 0x7f758806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f75903c9640 1 -- 172.21.15.72:0/1935156338 >> 172.21.15.72:0/1935156338 conn(0x7f75880cee70 msgr2=0x7f75880cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:22.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f75903c9640 1 -- 172.21.15.72:0/1935156338 shutdown_connections 2023-12-12T09:52:22.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.412+0000 7f75903c9640 1 -- 172.21.15.72:0/1935156338 wait complete. 2023-12-12T09:52:22.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.414+0000 7f75903c9640 1 Processor -- start 2023-12-12T09:52:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.414+0000 7f75903c9640 1 -- start start 2023-12-12T09:52:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f75903c9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758806a460 0x7f75881e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f75903c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 0x7f75880c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f75903c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 0x7f7588149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f75903c9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7588149f40 con 0x7f758806a460 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f75903c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7588147090 con 0x7f75880ce7e0 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f75903c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7588147370 con 0x7f758815af50 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f758e165640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758806a460 0x7f75881e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f758e165640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758806a460 0x7f75881e4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34320/0 (socket says 172.21.15.72:34320) 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.415+0000 7f758e165640 1 -- 172.21.15.72:0/1538022211 learned_addr learned my addr 172.21.15.72:0/1538022211 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f758e966640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 0x7f7588149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f758d964640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 0x7f75880c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f758e165640 1 -- 172.21.15.72:0/1538022211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 msgr2=0x7f7588149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f758e165640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 0x7f7588149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f758e165640 1 -- 172.21.15.72:0/1538022211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 msgr2=0x7f75880c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f758e165640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 0x7f75880c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f758e165640 1 -- 172.21.15.72:0/1538022211 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0094d0 con 0x7f758806a460 2023-12-12T09:52:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f758e165640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758806a460 0x7f75881e4a90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9b0 tx=0x7f758400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f757affd640 1 -- 172.21.15.72:0/1538022211 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584009670 con 0x7f758806a460 2023-12-12T09:52:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f757affd640 1 -- 172.21.15.72:0/1538022211 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758400c9a0 con 0x7f758806a460 2023-12-12T09:52:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.416+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7588147650 con 0x7f758806a460 2023-12-12T09:52:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.417+0000 7f757affd640 1 -- 172.21.15.72:0/1538022211 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400cd50 con 0x7f758806a460 2023-12-12T09:52:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.417+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7588147c10 con 0x7f758806a460 2023-12-12T09:52:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.418+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7588074450 con 0x7f758806a460 2023-12-12T09:52:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.419+0000 7f757affd640 1 -- 172.21.15.72:0/1538022211 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7584013470 con 0x7f758806a460 2023-12-12T09:52:22.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.420+0000 7f757affd640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f756c06ae40 0x7f756c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.420+0000 7f757affd640 1 -- 172.21.15.72:0/1538022211 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7584089b30 con 0x7f758806a460 2023-12-12T09:52:22.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.420+0000 7f758d964640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f756c06ae40 0x7f756c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.421+0000 7f758d964640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f756c06ae40 0x7f756c06d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f757c036000 tx=0x7f757c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:22.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.424+0000 7f757affd640 1 -- 172.21.15.72:0/1538022211 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7584054980 con 0x7f758806a460 2023-12-12T09:52:22.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.614+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75880ce1d0 con 0x7f758806a460 2023-12-12T09:52:22.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.615+0000 7f757affd640 1 -- 172.21.15.72:0/1538022211 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75840170a0 con 0x7f758806a460 2023-12-12T09:52:22.617 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f756c06ae40 msgr2=0x7f756c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f756c06ae40 0x7f756c06d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f757c036000 tx=0x7f757c03d040 comp rx=0 tx=0).stop 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758806a460 msgr2=0x7f75881e4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758806a460 0x7f75881e4a90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9b0 tx=0x7f758400ee70 comp rx=0 tx=0).stop 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 shutdown_connections 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f756c06ae40 0x7f756c06d300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f758815af50 0x7f7588149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75880ce7e0 0x7f75880c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 --2- 172.21.15.72:0/1538022211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758806a460 0x7f75881e4a90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 >> 172.21.15.72:0/1538022211 conn(0x7f75880cee70 msgr2=0x7f75880cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:22.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 shutdown_connections 2023-12-12T09:52:22.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.618+0000 7f75903c9640 1 -- 172.21.15.72:0/1538022211 wait complete. 2023-12-12T09:52:22.639 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:52:22.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.847+0000 7f6c9becf640 1 Processor -- start 2023-12-12T09:52:22.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.848+0000 7f6c9becf640 1 -- start start 2023-12-12T09:52:22.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.849+0000 7f6c9becf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c9406db50 0x7f6c94141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.849+0000 7f6c9becf640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9415b780 con 0x7f6c9406d770 2023-12-12T09:52:22.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.849+0000 7f6c9becf640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c940b2510 con 0x7f6c94141790 2023-12-12T09:52:22.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.849+0000 7f6c9becf640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c94141c20 con 0x7f6c9406db50 2023-12-12T09:52:22.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.849+0000 7f6c99c6b640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6c9406d770 0x7f6c940b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56650/0 (socket says 172.21.15.72:56650) 2023-12-12T09:52:22.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.849+0000 7f6c99c6b640 1 -- 172.21.15.72:0/1621483104 learned_addr learned my addr 172.21.15.72:0/1621483104 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:22.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c9946a640 1 --2- 172.21.15.72:0/1621483104 >> v2:172.21.15.113:3300/0 conn(0x7f6c9406db50 0x7f6c94141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294805554 0 0) 0x7f6c9415b780 con 0x7f6c9406d770 2023-12-12T09:52:22.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c80003520 con 0x7f6c9406d770 2023-12-12T09:52:22.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 596224917 0 0) 0x7f6c80003520 con 0x7f6c9406d770 2023-12-12T09:52:22.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 >> v2:172.21.15.113:3300/0 conn(0x7f6c9406db50 msgr2=0x7f6c94141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c98c69640 1 --2- 172.21.15.72:0/1621483104 >> v2:172.21.15.113:3300/0 conn(0x7f6c9406db50 0x7f6c94141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 >> v1:172.21.15.113:6789/0 conn(0x7f6c94141790 legacy=0x7f6c940cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9415b9d0 con 0x7f6c9406d770 2023-12-12T09:52:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.850+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f6c90001670 con 0x7f6c9406d770 2023-12-12T09:52:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 >> v1:172.21.15.72:6789/0 conn(0x7f6c9406d770 legacy=0x7f6c940b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c98c69640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c80003970 0x7f6c80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c98c69640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c800047c0 0x7f6c80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c98c69640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c80005180 0x7f6c8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94141c20 con 0x7f6c800047c0 2023-12-12T09:52:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8000c020 con 0x7f6c80005180 2023-12-12T09:52:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8000c1e0 con 0x7f6c80003970 2023-12-12T09:52:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9946a640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c800047c0 0x7f6c80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9a46c640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c80005180 0x7f6c8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c99c6b640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c80003970 0x7f6c80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9946a640 1 -- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c80003970 msgr2=0x7f6c80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9946a640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c80003970 0x7f6c80003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9946a640 1 -- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c80005180 msgr2=0x7f6c8000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:22.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9946a640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c80005180 0x7f6c8000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9946a640 1 -- 172.21.15.72:0/1621483104 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c80003520 con 0x7f6c800047c0 2023-12-12T09:52:22.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9a46c640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c80005180 0x7f6c8000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:22.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.851+0000 7f6c9a46c640 1 -- 172.21.15.72:0/1621483104 reap_dead start 2023-12-12T09:52:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.852+0000 7f6c9946a640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c800047c0 0x7f6c80004c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009ac0 tx=0x7f6c800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23be6fd2c8806bd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8402c6b0 con 0x7f6c800047c0 2023-12-12T09:52:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8402c810 con 0x7f6c800047c0 2023-12-12T09:52:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c98c69640 1 -- 172.21.15.72:0/1621483104 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8402cba0 con 0x7f6c800047c0 2023-12-12T09:52:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c9becf640 1 -- 172.21.15.72:0/1621483104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c800047c0 msgr2=0x7f6c80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c9becf640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c800047c0 0x7f6c80004c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009ac0 tx=0x7f6c800057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c9becf640 1 -- 172.21.15.72:0/1621483104 shutdown_connections 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c9becf640 1 --2- 172.21.15.72:0/1621483104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c800047c0 0x7f6c80004c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c9becf640 1 -- 172.21.15.72:0/1621483104 >> 172.21.15.72:0/1621483104 conn(0x7f6c940cb850 msgr2=0x7f6c940cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c9becf640 1 -- 172.21.15.72:0/1621483104 shutdown_connections 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.853+0000 7f6c9becf640 1 -- 172.21.15.72:0/1621483104 wait complete. 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.855+0000 7f6c9becf640 1 Processor -- start 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.855+0000 7f6c9becf640 1 -- start start 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.856+0000 7f6c9becf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c800047c0 0x7f6c941f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.856+0000 7f6c9becf640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c941447b0 0x7f6c941f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.856+0000 7f6c9becf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9415f0a0 0x7f6c94142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.856+0000 7f6c9becf640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941f37b0 con 0x7f6c941447b0 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.856+0000 7f6c9becf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941f3a90 con 0x7f6c9415f0a0 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.856+0000 7f6c9becf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941422d0 con 0x7f6c800047c0 2023-12-12T09:52:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c941447b0 0x7f6c941f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c941447b0 0x7f6c941f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34330/0 (socket says 172.21.15.72:34330) 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 -- 172.21.15.72:0/3598472536 learned_addr learned my addr 172.21.15.72:0/3598472536 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9a46c640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9415f0a0 0x7f6c94142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c99c6b640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c800047c0 0x7f6c941f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 -- 172.21.15.72:0/3598472536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c800047c0 msgr2=0x7f6c941f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c800047c0 0x7f6c941f2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 -- 172.21.15.72:0/3598472536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9415f0a0 msgr2=0x7f6c94142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9415f0a0 0x7f6c94142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 -- 172.21.15.72:0/3598472536 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c941425b0 con 0x7f6c941447b0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.857+0000 7f6c9946a640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c941447b0 0x7f6c941f4000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6c8402c250 tx=0x7f6c84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.858+0000 7f6c8a7fc640 1 -- 172.21.15.72:0/3598472536 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84003810 con 0x7f6c941447b0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.858+0000 7f6c8a7fc640 1 -- 172.21.15.72:0/3598472536 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c84003970 con 0x7f6c941447b0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.858+0000 7f6c8a7fc640 1 -- 172.21.15.72:0/3598472536 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84003d00 con 0x7f6c941447b0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.858+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c941f3d70 con 0x7f6c941447b0 2023-12-12T09:52:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.858+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941f3560 con 0x7f6c941447b0 2023-12-12T09:52:22.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.859+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c94142740 con 0x7f6c941447b0 2023-12-12T09:52:22.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.860+0000 7f6c8a7fc640 1 -- 172.21.15.72:0/3598472536 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c84034440 con 0x7f6c941447b0 2023-12-12T09:52:22.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.861+0000 7f6c8a7fc640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7806ae40 0x7f6c7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:22.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.861+0000 7f6c8a7fc640 1 -- 172.21.15.72:0/3598472536 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6c840abc10 con 0x7f6c941447b0 2023-12-12T09:52:22.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.861+0000 7f6c99c6b640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7806ae40 0x7f6c7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:22.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.862+0000 7f6c99c6b640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7806ae40 0x7f6c7806d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c90001200 tx=0x7f6c90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:22.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:22.865+0000 7f6c8a7fc640 1 -- 172.21.15.72:0/3598472536 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c84076a80 con 0x7f6c941447b0 2023-12-12T09:52:23.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.057+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c940745d0 con 0x7f6c941447b0 2023-12-12T09:52:23.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.058+0000 7f6c8a7fc640 1 -- 172.21.15.72:0/3598472536 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c8404c020 con 0x7f6c941447b0 2023-12-12T09:52:23.060 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:52:23.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7806ae40 msgr2=0x7f6c7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7806ae40 0x7f6c7806d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c90001200 tx=0x7f6c90009040 comp rx=0 tx=0).stop 2023-12-12T09:52:23.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c941447b0 msgr2=0x7f6c941f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c941447b0 0x7f6c941f4000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6c8402c250 tx=0x7f6c84004730 comp rx=0 tx=0).stop 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 shutdown_connections 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7806ae40 0x7f6c7806d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9415f0a0 0x7f6c94142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c941447b0 0x7f6c941f4000 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 --2- 172.21.15.72:0/3598472536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c800047c0 0x7f6c941f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.060+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 >> 172.21.15.72:0/3598472536 conn(0x7f6c940cb850 msgr2=0x7f6c940adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.061+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 shutdown_connections 2023-12-12T09:52:23.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.061+0000 7f6c9becf640 1 -- 172.21.15.72:0/3598472536 wait complete. 2023-12-12T09:52:23.080 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:52:23.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.292+0000 7f77c8940640 1 Processor -- start 2023-12-12T09:52:23.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.293+0000 7f77c8940640 1 -- start start 2023-12-12T09:52:23.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.294+0000 7f77c8940640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77c4155400 0x7f77c406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.294+0000 7f77c8940640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c40cf500 0x7f77c40ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.294+0000 7f77c8940640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c415b4a0 con 0x7f77c40ce150 2023-12-12T09:52:23.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.294+0000 7f77c8940640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c40adbb0 con 0x7f77c4155400 2023-12-12T09:52:23.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.294+0000 7f77c8940640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40ade90 con 0x7f77c40cf500 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.295+0000 7f77bbfff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f77c40ce150 0x7f77c40cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56664/0 (socket says 172.21.15.72:56664) 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.295+0000 7f77bbfff640 1 -- 172.21.15.72:0/2997440822 learned_addr learned my addr 172.21.15.72:0/2997440822 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.295+0000 7f77c359e640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c40cf500 0x7f77c40ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.295+0000 7f77c2d9d640 1 --2- 172.21.15.72:0/2997440822 >> v2:172.21.15.113:3300/0 conn(0x7f77c4155400 0x7f77c406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.295+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591423504 0 0) 0x7f77c415b4a0 con 0x7f77c40ce150 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.295+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b0003520 con 0x7f77c40ce150 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 71044136 0 0) 0x7f77b0003520 con 0x7f77c40ce150 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c40cf500 msgr2=0x7f77c40ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c40cf500 0x7f77c40ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 >> v2:172.21.15.113:3300/0 conn(0x7f77c4155400 msgr2=0x7f77c406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 --2- 172.21.15.72:0/2997440822 >> v2:172.21.15.113:3300/0 conn(0x7f77c4155400 0x7f77c406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40ae170 con 0x7f77c40ce150 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f77ac001670 con 0x7f77c40ce150 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 >> v1:172.21.15.72:6789/0 conn(0x7f77c40ce150 legacy=0x7f77c40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c40cf500 0x7f77b0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77b0003f40 0x7f77b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77b0004d00 0x7f77b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40adbb0 con 0x7f77b0003f40 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40ade90 con 0x7f77b0004d00 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b000b980 con 0x7f77c40cf500 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77bbfff640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77b0003f40 0x7f77b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c2d9d640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c40cf500 0x7f77b0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77c359e640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77b0004d00 0x7f77b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77bbfff640 1 -- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c40cf500 msgr2=0x7f77b0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77bbfff640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77c40cf500 0x7f77b0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77bbfff640 1 -- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77b0004d00 msgr2=0x7f77b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77bbfff640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77b0004d00 0x7f77b0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.296+0000 7f77bbfff640 1 -- 172.21.15.72:0/2997440822 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b0003520 con 0x7f77b0003f40 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.298+0000 7f77bbfff640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77b0003f40 0x7f77b00047c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c140 tx=0x7f77b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148cc7c97aee337a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.298+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac005ad0 con 0x7f77b0003f40 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.298+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac005c50 con 0x7f77b0003f40 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.298+0000 7f77c259c640 1 -- 172.21.15.72:0/2997440822 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02c810 con 0x7f77b0003f40 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.298+0000 7f77c8940640 1 -- 172.21.15.72:0/2997440822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77b0003f40 msgr2=0x7f77b00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.298+0000 7f77c8940640 1 --2- 172.21.15.72:0/2997440822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77b0003f40 0x7f77b00047c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c140 tx=0x7f77b00057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.298+0000 7f77c359e640 1 -- 172.21.15.72:0/2997440822 reap_dead start 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.299+0000 7f77c8940640 1 -- 172.21.15.72:0/2997440822 shutdown_connections 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.299+0000 7f77c8940640 1 -- 172.21.15.72:0/2997440822 >> 172.21.15.72:0/2997440822 conn(0x7f77c406a640 msgr2=0x7f77c406aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.299+0000 7f77c8940640 1 -- 172.21.15.72:0/2997440822 shutdown_connections 2023-12-12T09:52:23.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.299+0000 7f77c8940640 1 -- 172.21.15.72:0/2997440822 wait complete. 2023-12-12T09:52:23.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.301+0000 7f77c8940640 1 Processor -- start 2023-12-12T09:52:23.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.301+0000 7f77c8940640 1 -- start start 2023-12-12T09:52:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c8940640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77b0003f40 0x7f77c413bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c8940640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c413c080 0x7f77c40afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c8940640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c413a530 0x7f77c40b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c8940640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40b0640 con 0x7f77c413a530 2023-12-12T09:52:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c8940640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c413b6b0 con 0x7f77c413c080 2023-12-12T09:52:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c8940640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c414d910 con 0x7f77b0003f40 2023-12-12T09:52:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c359e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c413a530 0x7f77c40b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c359e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c413a530 0x7f77c40b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34354/0 (socket says 172.21.15.72:34354) 2023-12-12T09:52:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c359e640 1 -- 172.21.15.72:0/1099086471 learned_addr learned my addr 172.21.15.72:0/1099086471 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77bbfff640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c413c080 0x7f77c40afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c359e640 1 -- 172.21.15.72:0/1099086471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77b0003f40 msgr2=0x7f77c413bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c359e640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77b0003f40 0x7f77c413bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c359e640 1 -- 172.21.15.72:0/1099086471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c413c080 msgr2=0x7f77c40afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c359e640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c413c080 0x7f77c40afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.302+0000 7f77c359e640 1 -- 172.21.15.72:0/1099086471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c413aaa0 con 0x7f77c413a530 2023-12-12T09:52:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.303+0000 7f77c359e640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c413a530 0x7f77c40b0100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f77c406d6f0 tx=0x7f77bc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.303+0000 7f77bb7fe640 1 -- 172.21.15.72:0/1099086471 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc017400 con 0x7f77c413a530 2023-12-12T09:52:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.303+0000 7f77bb7fe640 1 -- 172.21.15.72:0/1099086471 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc011040 con 0x7f77c413a530 2023-12-12T09:52:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.303+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c413ad80 con 0x7f77c413a530 2023-12-12T09:52:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.303+0000 7f77bb7fe640 1 -- 172.21.15.72:0/1099086471 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0177c0 con 0x7f77c413a530 2023-12-12T09:52:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.303+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c413b340 con 0x7f77c413a530 2023-12-12T09:52:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.304+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c4074450 con 0x7f77c413a530 2023-12-12T09:52:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.305+0000 7f77bb7fe640 1 -- 172.21.15.72:0/1099086471 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77bc0203b0 con 0x7f77c413a530 2023-12-12T09:52:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.306+0000 7f77bb7fe640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f77a406ae70 0x7f77a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.306+0000 7f77bb7fe640 1 -- 172.21.15.72:0/1099086471 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f77bc089a30 con 0x7f77c413a530 2023-12-12T09:52:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.307+0000 7f77c2d9d640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f77a406ae70 0x7f77a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.307+0000 7f77c2d9d640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f77a406ae70 0x7f77a406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f77b40099d0 tx=0x7f77b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:23.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.311+0000 7f77bb7fe640 1 -- 172.21.15.72:0/1099086471 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77bc0547f0 con 0x7f77c413a530 2023-12-12T09:52:23.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.502+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77c413b7f0 con 0x7f77c413a530 2023-12-12T09:52:23.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.503+0000 7f77bb7fe640 1 -- 172.21.15.72:0/1099086471 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77bc057e10 con 0x7f77c413a530 2023-12-12T09:52:23.505 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:52:23.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f77a406ae70 msgr2=0x7f77a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f77a406ae70 0x7f77a406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f77b40099d0 tx=0x7f77b4008040 comp rx=0 tx=0).stop 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c413a530 msgr2=0x7f77c40b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c413a530 0x7f77c40b0100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f77c406d6f0 tx=0x7f77bc00f4a0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 shutdown_connections 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f77a406ae70 0x7f77a406d330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c413a530 0x7f77c40b0100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c413c080 0x7f77c40afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 --2- 172.21.15.72:0/1099086471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77b0003f40 0x7f77c413bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.505+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 >> 172.21.15.72:0/1099086471 conn(0x7f77c406a640 msgr2=0x7f77c4159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.506+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 shutdown_connections 2023-12-12T09:52:23.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.506+0000 7f77c8940640 1 -- 172.21.15.72:0/1099086471 wait complete. 2023-12-12T09:52:23.527 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:52:23.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.746+0000 7f31f751d640 1 Processor -- start 2023-12-12T09:52:23.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.747+0000 7f31f751d640 1 -- start start 2023-12-12T09:52:23.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.748+0000 7f31f751d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f31f006db50 0x7f31f0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.748+0000 7f31f751d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31f0141790 0x7f31f00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.748+0000 7f31f751d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f015b780 con 0x7f31f006d770 2023-12-12T09:52:23.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.748+0000 7f31f751d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f0141c20 con 0x7f31f006db50 2023-12-12T09:52:23.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.748+0000 7f31f751d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00b2510 con 0x7f31f0141790 2023-12-12T09:52:23.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.748+0000 7f31f52b9640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f31f006d770 0x7f31f00b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56666/0 (socket says 172.21.15.72:56666) 2023-12-12T09:52:23.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.748+0000 7f31f52b9640 1 -- 172.21.15.72:0/2455431087 learned_addr learned my addr 172.21.15.72:0/2455431087 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:23.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31f4ab8640 1 --2- 172.21.15.72:0/2455431087 >> v2:172.21.15.113:3300/0 conn(0x7f31f006db50 0x7f31f0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31f5aba640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31f0141790 0x7f31f00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394687638 0 0) 0x7f31f015b780 con 0x7f31f006d770 2023-12-12T09:52:23.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d8003520 con 0x7f31f006d770 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110628936 0 0) 0x7f31d8003520 con 0x7f31f006d770 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31f0141790 msgr2=0x7f31f00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31f0141790 0x7f31f00cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 >> v2:172.21.15.113:3300/0 conn(0x7f31f006db50 msgr2=0x7f31f0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 --2- 172.21.15.72:0/2455431087 >> v2:172.21.15.113:3300/0 conn(0x7f31f006db50 0x7f31f0141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f015b9d0 con 0x7f31f006d770 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f31e00019f0 con 0x7f31f006d770 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 >> v1:172.21.15.72:6789/0 conn(0x7f31f006d770 legacy=0x7f31f00b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31d8003970 0x7f31d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31f0141790 0x7f31d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31d8004d00 0x7f31d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0141c20 con 0x7f31d8004d00 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00b2510 con 0x7f31d8003970 2023-12-12T09:52:23.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.749+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800b980 con 0x7f31f0141790 2023-12-12T09:52:23.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.750+0000 7f31f5aba640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31d8004d00 0x7f31d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.750+0000 7f31f52b9640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31d8003970 0x7f31d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.750+0000 7f31f4ab8640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31f0141790 0x7f31d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.750+0000 7f31f5aba640 1 -- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31f0141790 msgr2=0x7f31d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:23.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.750+0000 7f31f5aba640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31f0141790 0x7f31d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.750+0000 7f31f5aba640 1 -- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31d8003970 msgr2=0x7f31d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:23.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.750+0000 7f31f5aba640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31d8003970 0x7f31d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.750+0000 7f31f5aba640 1 -- 172.21.15.72:0/2455431087 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8003520 con 0x7f31d8004d00 2023-12-12T09:52:23.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.751+0000 7f31f5aba640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31d8004d00 0x7f31d8005180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009ac0 tx=0x7f31d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc358be64e1538f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.751+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e802c6b0 con 0x7f31d8004d00 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.751+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e802c810 con 0x7f31d8004d00 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.751+0000 7f31effff640 1 -- 172.21.15.72:0/2455431087 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e802cba0 con 0x7f31d8004d00 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.752+0000 7f31f751d640 1 -- 172.21.15.72:0/2455431087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31d8004d00 msgr2=0x7f31d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.752+0000 7f31f751d640 1 --2- 172.21.15.72:0/2455431087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31d8004d00 0x7f31d8005180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009ac0 tx=0x7f31d80057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.752+0000 7f31f5aba640 1 -- 172.21.15.72:0/2455431087 reap_dead start 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.752+0000 7f31f751d640 1 -- 172.21.15.72:0/2455431087 shutdown_connections 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.752+0000 7f31f751d640 1 -- 172.21.15.72:0/2455431087 >> 172.21.15.72:0/2455431087 conn(0x7f31f00cb850 msgr2=0x7f31f00cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.752+0000 7f31f751d640 1 -- 172.21.15.72:0/2455431087 shutdown_connections 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.752+0000 7f31f751d640 1 -- 172.21.15.72:0/2455431087 wait complete. 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.754+0000 7f31f751d640 1 Processor -- start 2023-12-12T09:52:23.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.754+0000 7f31f751d640 1 -- start start 2023-12-12T09:52:23.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f751d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31d8004d00 0x7f31f0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f751d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31f013e2e0 0x7f31f013e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f751d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f0159aa0 0x7f31f0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f751d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00707d0 con 0x7f31f013e2e0 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f751d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f013eca0 con 0x7f31f0159aa0 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f751d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f015a810 con 0x7f31d8004d00 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f4ab8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31f013e2e0 0x7f31f013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f4ab8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31f013e2e0 0x7f31f013e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34384/0 (socket says 172.21.15.72:34384) 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f4ab8640 1 -- 172.21.15.72:0/759177770 learned_addr learned my addr 172.21.15.72:0/759177770 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.755+0000 7f31f52b9640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31d8004d00 0x7f31f0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31f5aba640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f0159aa0 0x7f31f0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31f4ab8640 1 -- 172.21.15.72:0/759177770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31d8004d00 msgr2=0x7f31f0155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31f4ab8640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31d8004d00 0x7f31f0155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31f4ab8640 1 -- 172.21.15.72:0/759177770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f0159aa0 msgr2=0x7f31f0159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31f4ab8640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f0159aa0 0x7f31f0159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31f4ab8640 1 -- 172.21.15.72:0/759177770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f015aaf0 con 0x7f31f013e2e0 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31f4ab8640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31f013e2e0 0x7f31f013e760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f31e40116d0 tx=0x7f31e4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31edffb640 1 -- 172.21.15.72:0/759177770 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400f9c0 con 0x7f31f013e2e0 2023-12-12T09:52:23.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31edffb640 1 -- 172.21.15.72:0/759177770 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400fb20 con 0x7f31f013e2e0 2023-12-12T09:52:23.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31edffb640 1 -- 172.21.15.72:0/759177770 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400fe90 con 0x7f31f013e2e0 2023-12-12T09:52:23.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.756+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f00703f0 con 0x7f31f013e2e0 2023-12-12T09:52:23.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.757+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f0156110 con 0x7f31f013e2e0 2023-12-12T09:52:23.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.758+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f0070250 con 0x7f31f013e2e0 2023-12-12T09:52:23.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.759+0000 7f31edffb640 1 -- 172.21.15.72:0/759177770 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f31e4015460 con 0x7f31f013e2e0 2023-12-12T09:52:23.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.760+0000 7f31edffb640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f31dc06ae40 0x7f31dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:23.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.760+0000 7f31edffb640 1 -- 172.21.15.72:0/759177770 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f31e4089e60 con 0x7f31f013e2e0 2023-12-12T09:52:23.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.760+0000 7f31f52b9640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f31dc06ae40 0x7f31dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:23.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.761+0000 7f31f52b9640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f31dc06ae40 0x7f31dc06d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f31e00013d0 tx=0x7f31e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:23.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.764+0000 7f31edffb640 1 -- 172.21.15.72:0/759177770 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f31f0070250 con 0x7f31f013e2e0 2023-12-12T09:52:23.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.955+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f31f0070250 con 0x7f31f013e2e0 2023-12-12T09:52:23.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.956+0000 7f31edffb640 1 -- 172.21.15.72:0/759177770 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f31f0070250 con 0x7f31f013e2e0 2023-12-12T09:52:23.958 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:52:23.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f31dc06ae40 msgr2=0x7f31dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f31dc06ae40 0x7f31dc06d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f31e00013d0 tx=0x7f31e0009040 comp rx=0 tx=0).stop 2023-12-12T09:52:23.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31f013e2e0 msgr2=0x7f31f013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:23.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31f013e2e0 0x7f31f013e760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f31e40116d0 tx=0x7f31e4011b90 comp rx=0 tx=0).stop 2023-12-12T09:52:23.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 shutdown_connections 2023-12-12T09:52:23.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f31dc06ae40 0x7f31dc06d300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31f0159aa0 0x7f31f0159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31f013e2e0 0x7f31f013e760 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 --2- 172.21.15.72:0/759177770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31d8004d00 0x7f31f0155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:23.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 >> 172.21.15.72:0/759177770 conn(0x7f31f00cb850 msgr2=0x7f31f015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:23.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 shutdown_connections 2023-12-12T09:52:23.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:23.959+0000 7f31f751d640 1 -- 172.21.15.72:0/759177770 wait complete. 2023-12-12T09:52:23.980 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:52:24.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.191+0000 7f72a8d2e640 1 Processor -- start 2023-12-12T09:52:24.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.191+0000 7f72a8d2e640 1 -- start start 2023-12-12T09:52:24.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a8d2e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72a40cde70 0x7f72a406d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a8d2e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a415e840 con 0x7f72a415d380 2023-12-12T09:52:24.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a8d2e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a415dcc0 con 0x7f72a40c58b0 2023-12-12T09:52:24.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a8d2e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a415dfa0 con 0x7f72a40cde70 2023-12-12T09:52:24.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a37fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f72a415d380 0x7f72a40c9d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56670/0 (socket says 172.21.15.72:56670) 2023-12-12T09:52:24.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a37fe640 1 -- 172.21.15.72:0/1711245717 learned_addr learned my addr 172.21.15.72:0/1711245717 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:24.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a2ffd640 1 --2- 172.21.15.72:0/1711245717 >> v2:172.21.15.113:3300/0 conn(0x7f72a40cde70 0x7f72a406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476714146 0 0) 0x7f72a415e840 con 0x7f72a415d380 2023-12-12T09:52:24.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.193+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f728c003520 con 0x7f72a415d380 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1761103516 0 0) 0x7f728c003520 con 0x7f72a415d380 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 >> v2:172.21.15.113:3300/0 conn(0x7f72a40cde70 msgr2=0x7f72a406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 --2- 172.21.15.72:0/1711245717 >> v2:172.21.15.113:3300/0 conn(0x7f72a40cde70 0x7f72a406d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 >> v1:172.21.15.113:6789/0 conn(0x7f72a40c58b0 legacy=0x7f72a40c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a415e280 con 0x7f72a415d380 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f7294001670 con 0x7f72a415d380 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 >> v1:172.21.15.72:6789/0 conn(0x7f72a415d380 legacy=0x7f72a40c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f728c0039c0 0x7f728c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f728c0047c0 0x7f728c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f728c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a415dfa0 con 0x7f728c005180 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c00c050 con 0x7f728c0039c0 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c00c210 con 0x7f728c0047c0 2023-12-12T09:52:24.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a3fff640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f728c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a2ffd640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f728c0047c0 0x7f728c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a37fe640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f728c0039c0 0x7f728c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a3fff640 1 -- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f728c0047c0 msgr2=0x7f728c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a3fff640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f728c0047c0 0x7f728c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.194+0000 7f72a3fff640 1 -- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f728c0039c0 msgr2=0x7f728c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.195+0000 7f72a3fff640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f728c0039c0 0x7f728c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.195+0000 7f72a3fff640 1 -- 172.21.15.72:0/1711245717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c003520 con 0x7f728c005180 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.195+0000 7f72a37fe640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f728c0039c0 0x7f728c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.195+0000 7f72a3fff640 1 -- 172.21.15.72:0/1711245717 reap_dead start 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.196+0000 7f72a3fff640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f728c00b980 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f729c0044b0 tx=0x7f728c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4109a27cbdbfca server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.196+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c006d50 con 0x7f728c005180 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.196+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c007020 con 0x7f728c005180 2023-12-12T09:52:24.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.196+0000 7f72a27fc640 1 -- 172.21.15.72:0/1711245717 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c0073d0 con 0x7f728c005180 2023-12-12T09:52:24.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.196+0000 7f72a8d2e640 1 -- 172.21.15.72:0/1711245717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 msgr2=0x7f728c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.196+0000 7f72a8d2e640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f728c00b980 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f729c0044b0 tx=0x7f728c0057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.196+0000 7f72a8d2e640 1 -- 172.21.15.72:0/1711245717 shutdown_connections 2023-12-12T09:52:24.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.196+0000 7f72a8d2e640 1 --2- 172.21.15.72:0/1711245717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f728c00b980 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.197+0000 7f72a8d2e640 1 -- 172.21.15.72:0/1711245717 >> 172.21.15.72:0/1711245717 conn(0x7f72a40b0b90 msgr2=0x7f72a40b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:24.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.197+0000 7f72a8d2e640 1 -- 172.21.15.72:0/1711245717 shutdown_connections 2023-12-12T09:52:24.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.197+0000 7f72a8d2e640 1 -- 172.21.15.72:0/1711245717 wait complete. 2023-12-12T09:52:24.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.198+0000 7f72a8d2e640 1 Processor -- start 2023-12-12T09:52:24.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.199+0000 7f72a8d2e640 1 -- start start 2023-12-12T09:52:24.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.199+0000 7f72a8d2e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f72a40ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.199+0000 7f72a8d2e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a4153cb0 0x7f72a406d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a8d2e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72a4071700 0x7f72a406dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a8d2e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40ae1a0 con 0x7f728c005180 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a8d2e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40adbe0 con 0x7f72a4153cb0 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a8d2e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40adec0 con 0x7f72a4071700 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f72a40ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f72a40ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34392/0 (socket says 172.21.15.72:34392) 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 -- 172.21.15.72:0/3801435058 learned_addr learned my addr 172.21.15.72:0/3801435058 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 -- 172.21.15.72:0/3801435058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72a4071700 msgr2=0x7f72a406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72a4071700 0x7f72a406dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 -- 172.21.15.72:0/3801435058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a4153cb0 msgr2=0x7f72a406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a4153cb0 0x7f72a406d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 -- 172.21.15.72:0/3801435058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a415aaa0 con 0x7f728c005180 2023-12-12T09:52:24.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.200+0000 7f72a37fe640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f72a40ad2d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f729400d720 tx=0x7f729400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:24.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.201+0000 7f728bfff640 1 -- 172.21.15.72:0/3801435058 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72940074e0 con 0x7f728c005180 2023-12-12T09:52:24.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.201+0000 7f728bfff640 1 -- 172.21.15.72:0/3801435058 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294007640 con 0x7f728c005180 2023-12-12T09:52:24.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.201+0000 7f728bfff640 1 -- 172.21.15.72:0/3801435058 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294004a00 con 0x7f728c005180 2023-12-12T09:52:24.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.201+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a415ad80 con 0x7f728c005180 2023-12-12T09:52:24.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.201+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a40afb50 con 0x7f728c005180 2023-12-12T09:52:24.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.202+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a415af10 con 0x7f728c005180 2023-12-12T09:52:24.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.203+0000 7f728bfff640 1 -- 172.21.15.72:0/3801435058 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7294004b60 con 0x7f728c005180 2023-12-12T09:52:24.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.204+0000 7f728bfff640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f729006ae40 0x7f729006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.204+0000 7f728bfff640 1 -- 172.21.15.72:0/3801435058 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7294089170 con 0x7f728c005180 2023-12-12T09:52:24.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.204+0000 7f72a2ffd640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f729006ae40 0x7f729006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.205+0000 7f72a2ffd640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f729006ae40 0x7f729006d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72a415c820 tx=0x7f7298008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:24.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.209+0000 7f728bfff640 1 -- 172.21.15.72:0/3801435058 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7294053f20 con 0x7f728c005180 2023-12-12T09:52:24.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.411+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f72a40673f0 con 0x7f728c005180 2023-12-12T09:52:24.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.412+0000 7f728bfff640 1 -- 172.21.15.72:0/3801435058 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7294057540 con 0x7f728c005180 2023-12-12T09:52:24.414 INFO:teuthology.orchestra.run.smithi072.stderr:listed 0 entries 2023-12-12T09:52:24.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f729006ae40 msgr2=0x7f729006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f729006ae40 0x7f729006d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72a415c820 tx=0x7f7298008040 comp rx=0 tx=0).stop 2023-12-12T09:52:24.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 msgr2=0x7f72a40ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f72a40ad2d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f729400d720 tx=0x7f729400dbe0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 shutdown_connections 2023-12-12T09:52:24.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f729006ae40 0x7f729006d300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72a4071700 0x7f72a406dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a4153cb0 0x7f72a406d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 --2- 172.21.15.72:0/3801435058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f728c005180 0x7f72a40ad2d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 >> 172.21.15.72:0/3801435058 conn(0x7f72a40b0b90 msgr2=0x7f72a4157240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:24.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 shutdown_connections 2023-12-12T09:52:24.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.415+0000 7f72a8d2e640 1 -- 172.21.15.72:0/3801435058 wait complete. 2023-12-12T09:52:24.434 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:52:24.434 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:52:24.486 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:24.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.699+0000 7fd88904b640 1 Processor -- start 2023-12-12T09:52:24.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.699+0000 7fd88904b640 1 -- start start 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.700+0000 7fd88904b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd8840ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.700+0000 7fd88904b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd8840713b0 0x7fd8840b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.700+0000 7fd88904b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd884070070 con 0x7fd884070750 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.700+0000 7fd88904b640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8840705f0 con 0x7fd8840713b0 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.700+0000 7fd88904b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd884070310 con 0x7fd884159fd0 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd88884a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd8840713b0 0x7fd8840b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd88884a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd8840713b0 0x7fd8840b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34402/0 (socket says 172.21.15.72:34402) 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd88884a640 1 -- 172.21.15.72:0/1817305029 learned_addr learned my addr 172.21.15.72:0/1817305029 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:24.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd87bfff640 1 --1- 172.21.15.72:0/1817305029 >> v1:172.21.15.72:6789/0 conn(0x7fd884070750 0x7fd884070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56680/0 (socket says 172.21.15.72:56680) 2023-12-12T09:52:24.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd883fff640 1 --2- 172.21.15.72:0/1817305029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd8840ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd88884a640 1 -- 172.21.15.72:0/1817305029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 msgr2=0x7fd8840ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:24.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd88884a640 1 --2- 172.21.15.72:0/1817305029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd8840ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd88884a640 1 -- 172.21.15.72:0/1817305029 >> v1:172.21.15.72:6789/0 conn(0x7fd884070750 legacy=0x7fd884070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd88884a640 1 -- 172.21.15.72:0/1817305029 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840b1d00 con 0x7fd8840713b0 2023-12-12T09:52:24.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.701+0000 7fd883fff640 1 --2- 172.21.15.72:0/1817305029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd8840ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.702+0000 7fd88884a640 1 --2- 172.21.15.72:0/1817305029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8840713b0 0x7fd8840b27e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009e50 tx=0x7fd87c02bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5218c155e6183fab server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd8837fe640 1 -- 172.21.15.72:0/1817305029 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c02cb80 con 0x7fd8840713b0 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd8837fe640 1 -- 172.21.15.72:0/1817305029 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c02cce0 con 0x7fd8840713b0 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd8837fe640 1 -- 172.21.15.72:0/1817305029 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c034610 con 0x7fd8840713b0 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd88904b640 1 -- 172.21.15.72:0/1817305029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8840713b0 msgr2=0x7fd8840b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd88904b640 1 --2- 172.21.15.72:0/1817305029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8840713b0 0x7fd8840b27e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009e50 tx=0x7fd87c02bfe0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd88904b640 1 -- 172.21.15.72:0/1817305029 shutdown_connections 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd88904b640 1 --2- 172.21.15.72:0/1817305029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8840713b0 0x7fd8840b27e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd88904b640 1 --2- 172.21.15.72:0/1817305029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd8840ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd88904b640 1 -- 172.21.15.72:0/1817305029 >> 172.21.15.72:0/1817305029 conn(0x7fd8840b0860 msgr2=0x7fd8840b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:24.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd88904b640 1 -- 172.21.15.72:0/1817305029 shutdown_connections 2023-12-12T09:52:24.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.703+0000 7fd88904b640 1 -- 172.21.15.72:0/1817305029 wait complete. 2023-12-12T09:52:24.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.705+0000 7fd88904b640 1 Processor -- start 2023-12-12T09:52:24.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.705+0000 7fd88904b640 1 -- start start 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd88904b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd884070750 0x7fd8841552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd88904b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8840713b0 0x7fd88415cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd88904b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd88415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd88904b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88415d900 con 0x7fd884070750 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd88904b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88415a850 con 0x7fd8840713b0 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd88904b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd884099b20 con 0x7fd884159fd0 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd883fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd884070750 0x7fd8841552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd883fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd884070750 0x7fd8841552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34408/0 (socket says 172.21.15.72:34408) 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.706+0000 7fd883fff640 1 -- 172.21.15.72:0/3023970482 learned_addr learned my addr 172.21.15.72:0/3023970482 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd88884a640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd88415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd87bfff640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8840713b0 0x7fd88415cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd883fff640 1 -- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 msgr2=0x7fd88415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd883fff640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd88415d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd883fff640 1 -- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8840713b0 msgr2=0x7fd88415cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd883fff640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8840713b0 0x7fd88415cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd883fff640 1 -- 172.21.15.72:0/3023970482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c009b30 con 0x7fd884070750 2023-12-12T09:52:24.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd87bfff640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8840713b0 0x7fd88415cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:24.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd883fff640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd884070750 0x7fd8841552c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c7f0 tx=0x7fd87400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:24.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd8817fa640 1 -- 172.21.15.72:0/3023970482 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874007820 con 0x7fd884070750 2023-12-12T09:52:24.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd8817fa640 1 -- 172.21.15.72:0/3023970482 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd874007980 con 0x7fd884070750 2023-12-12T09:52:24.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.707+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd884099e00 con 0x7fd884070750 2023-12-12T09:52:24.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.708+0000 7fd8817fa640 1 -- 172.21.15.72:0/3023970482 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874007cf0 con 0x7fd884070750 2023-12-12T09:52:24.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.708+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88409a3c0 con 0x7fd884070750 2023-12-12T09:52:24.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.709+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd884074450 con 0x7fd884070750 2023-12-12T09:52:24.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.710+0000 7fd8817fa640 1 -- 172.21.15.72:0/3023970482 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8740113f0 con 0x7fd884070750 2023-12-12T09:52:24.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.710+0000 7fd8817fa640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd86406ae40 0x7fd86406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:24.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.711+0000 7fd8817fa640 1 -- 172.21.15.72:0/3023970482 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd874089c20 con 0x7fd884070750 2023-12-12T09:52:24.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.711+0000 7fd87bfff640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd86406ae40 0x7fd86406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:24.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.711+0000 7fd87bfff640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd86406ae40 0x7fd86406d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd884155c10 tx=0x7fd86c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:24.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.715+0000 7fd8817fa640 1 -- 172.21.15.72:0/3023970482 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8740549d0 con 0x7fd884070750 2023-12-12T09:52:24.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.906+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8840ad9a0 con 0x7fd884070750 2023-12-12T09:52:24.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.909+0000 7fd8817fa640 1 -- 172.21.15.72:0/3023970482 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fd87401f090 con 0x7fd884070750 2023-12-12T09:52:24.911 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:24.911 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.911+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd86406ae40 msgr2=0x7fd86406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.911+0000 7fd88904b640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd86406ae40 0x7fd86406d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd884155c10 tx=0x7fd86c009040 comp rx=0 tx=0).stop 2023-12-12T09:52:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.911+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd884070750 msgr2=0x7fd8841552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.911+0000 7fd88904b640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd884070750 0x7fd8841552c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c7f0 tx=0x7fd87400ccb0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.912+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 shutdown_connections 2023-12-12T09:52:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.912+0000 7fd88904b640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd86406ae40 0x7fd86406d300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.912+0000 7fd88904b640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd884159fd0 0x7fd88415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.912+0000 7fd88904b640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8840713b0 0x7fd88415cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.912+0000 7fd88904b640 1 --2- 172.21.15.72:0/3023970482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd884070750 0x7fd8841552c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.912+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 >> 172.21.15.72:0/3023970482 conn(0x7fd8840b0860 msgr2=0x7fd88406d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.912+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 shutdown_connections 2023-12-12T09:52:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:24.912+0000 7fd88904b640 1 -- 172.21.15.72:0/3023970482 wait complete. 2023-12-12T09:52:24.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:52:24.934 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:52:25.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.138+0000 7f55a9a89640 1 Processor -- start 2023-12-12T09:52:25.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.138+0000 7f55a9a89640 1 -- start start 2023-12-12T09:52:25.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.139+0000 7f55a9a89640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55a415af50 0x7f55a406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.140+0000 7f55a9a89640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.140+0000 7f55a9a89640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a4071cd0 con 0x7f55a40ce7e0 2023-12-12T09:52:25.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.140+0000 7f55a9a89640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a40717e0 con 0x7f55a415af50 2023-12-12T09:52:25.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.140+0000 7f55a9a89640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a406ae10 con 0x7f55a406a460 2023-12-12T09:52:25.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.140+0000 7f55a3fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.140+0000 7f55a37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55a415af50 0x7f55a406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.140+0000 7f55a3fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a406a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:60518/0 (socket says 172.21.15.72:60518) 2023-12-12T09:52:25.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.140+0000 7f55a3fff640 1 -- 172.21.15.72:0/519901240 learned_addr learned my addr 172.21.15.72:0/519901240 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:25.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.141+0000 7f55a37fe640 1 -- 172.21.15.72:0/519901240 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 msgr2=0x7f55a406a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:25.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.141+0000 7f55a37fe640 1 --2- 172.21.15.72:0/519901240 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a406a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.141+0000 7f55a37fe640 1 -- 172.21.15.72:0/519901240 >> v1:172.21.15.113:6789/0 conn(0x7f55a40ce7e0 legacy=0x7f55a415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.141+0000 7f55a37fe640 1 -- 172.21.15.72:0/519901240 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a406c6b0 con 0x7f55a415af50 2023-12-12T09:52:25.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.143+0000 7f55a37fe640 1 --2- 172.21.15.72:0/519901240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 0x7f55a406c170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55940097a0 tx=0x7f559402b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5de92847f503e3bd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:25.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.144+0000 7f55a27fc640 1 -- 172.21.15.72:0/519901240 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559402f050 con 0x7f55a415af50 2023-12-12T09:52:25.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.144+0000 7f55a27fc640 1 -- 172.21.15.72:0/519901240 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5594031700 con 0x7f55a415af50 2023-12-12T09:52:25.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.144+0000 7f55a27fc640 1 -- 172.21.15.72:0/519901240 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594031a90 con 0x7f55a415af50 2023-12-12T09:52:25.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.144+0000 7f55a9a89640 1 -- 172.21.15.72:0/519901240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 msgr2=0x7f55a406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.144+0000 7f55a9a89640 1 --2- 172.21.15.72:0/519901240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 0x7f55a406c170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55940097a0 tx=0x7f559402b7a0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.145+0000 7f55a9a89640 1 -- 172.21.15.72:0/519901240 shutdown_connections 2023-12-12T09:52:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.145+0000 7f55a9a89640 1 --2- 172.21.15.72:0/519901240 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a406a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.145+0000 7f55a9a89640 1 --2- 172.21.15.72:0/519901240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 0x7f55a406c170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.145+0000 7f55a9a89640 1 -- 172.21.15.72:0/519901240 >> 172.21.15.72:0/519901240 conn(0x7f55a40cee70 msgr2=0x7f55a40cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.145+0000 7f55a9a89640 1 -- 172.21.15.72:0/519901240 shutdown_connections 2023-12-12T09:52:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.145+0000 7f55a9a89640 1 -- 172.21.15.72:0/519901240 wait complete. 2023-12-12T09:52:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.147+0000 7f55a9a89640 1 Processor -- start 2023-12-12T09:52:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.147+0000 7f55a9a89640 1 -- start start 2023-12-12T09:52:25.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.148+0000 7f55a9a89640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a41447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.148+0000 7f55a9a89640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a40ce7e0 0x7f55a40b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.148+0000 7f55a9a89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 0x7f55a409ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.148+0000 7f55a9a89640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40b32c0 con 0x7f55a40ce7e0 2023-12-12T09:52:25.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.148+0000 7f55a9a89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4099e90 con 0x7f55a415af50 2023-12-12T09:52:25.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.148+0000 7f55a9a89640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40b2d00 con 0x7f55a406a460 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a40ce7e0 0x7f55a40b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a40ce7e0 0x7f55a40b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53596/0 (socket says 172.21.15.72:53596) 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a2ffd640 1 -- 172.21.15.72:0/2441534531 learned_addr learned my addr 172.21.15.72:0/2441534531 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a37fe640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a41447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a3fff640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 0x7f55a409ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a2ffd640 1 -- 172.21.15.72:0/2441534531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 msgr2=0x7f55a41447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a2ffd640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a41447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a2ffd640 1 -- 172.21.15.72:0/2441534531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 msgr2=0x7f55a409ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a2ffd640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 0x7f55a409ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.149+0000 7f55a2ffd640 1 -- 172.21.15.72:0/2441534531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5594009400 con 0x7f55a40ce7e0 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.150+0000 7f55a2ffd640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a40ce7e0 0x7f55a40b2340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f559800c3c0 tx=0x7f559800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:25.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.150+0000 7f558bfff640 1 -- 172.21.15.72:0/2441534531 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55980052d0 con 0x7f55a40ce7e0 2023-12-12T09:52:25.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.150+0000 7f558bfff640 1 -- 172.21.15.72:0/2441534531 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598005450 con 0x7f55a40ce7e0 2023-12-12T09:52:25.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.150+0000 7f558bfff640 1 -- 172.21.15.72:0/2441534531 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598007260 con 0x7f55a40ce7e0 2023-12-12T09:52:25.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.150+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a40b2fe0 con 0x7f55a40ce7e0 2023-12-12T09:52:25.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.150+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a4144ce0 con 0x7f55a40ce7e0 2023-12-12T09:52:25.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.151+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a409a000 con 0x7f55a40ce7e0 2023-12-12T09:52:25.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.152+0000 7f558bfff640 1 -- 172.21.15.72:0/2441534531 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f55980073c0 con 0x7f55a40ce7e0 2023-12-12T09:52:25.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.153+0000 7f558bfff640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f559006ae70 0x7f559006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.153+0000 7f558bfff640 1 -- 172.21.15.72:0/2441534531 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5598088c20 con 0x7f55a40ce7e0 2023-12-12T09:52:25.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.153+0000 7f55a37fe640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f559006ae70 0x7f559006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.154+0000 7f55a37fe640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f559006ae70 0x7f559006d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5594035000 tx=0x7f5594008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:25.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.157+0000 7f558bfff640 1 -- 172.21.15.72:0/2441534531 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5598053a70 con 0x7f55a40ce7e0 2023-12-12T09:52:25.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.345+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55a4074450 con 0x7f55a40ce7e0 2023-12-12T09:52:25.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.345+0000 7f558bfff640 1 -- 172.21.15.72:0/2441534531 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5598057090 con 0x7f55a40ce7e0 2023-12-12T09:52:25.347 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f559006ae70 msgr2=0x7f559006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f559006ae70 0x7f559006d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5594035000 tx=0x7f5594008030 comp rx=0 tx=0).stop 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a40ce7e0 msgr2=0x7f55a40b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a40ce7e0 0x7f55a40b2340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f559800c3c0 tx=0x7f559800c880 comp rx=0 tx=0).stop 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 shutdown_connections 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f559006ae70 0x7f559006d330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55a415af50 0x7f55a409ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a40ce7e0 0x7f55a40b2340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 --2- 172.21.15.72:0/2441534531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a406a460 0x7f55a41447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 >> 172.21.15.72:0/2441534531 conn(0x7f55a40cee70 msgr2=0x7f55a40c4770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 shutdown_connections 2023-12-12T09:52:25.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.348+0000 7f55a9a89640 1 -- 172.21.15.72:0/2441534531 wait complete. 2023-12-12T09:52:25.368 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:52:25.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.584+0000 7fc8a1e9b640 1 Processor -- start 2023-12-12T09:52:25.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.584+0000 7fc8a1e9b640 1 -- start start 2023-12-12T09:52:25.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc8a1e9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc89c155a50 0x7fc89c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc8a1e9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0b32a0 0x7fc89c0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc8a1e9b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0b5a10 con 0x7fc89c15b970 2023-12-12T09:52:25.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc8a1e9b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0b62b0 con 0x7fc89c155a50 2023-12-12T09:52:25.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc8a1e9b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b5cf0 con 0x7fc89c0b32a0 2023-12-12T09:52:25.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc89affd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc89c15b970 0x7fc89c0b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42054/0 (socket says 172.21.15.72:42054) 2023-12-12T09:52:25.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc89affd640 1 -- 172.21.15.72:0/1985827735 learned_addr learned my addr 172.21.15.72:0/1985827735 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc89b7fe640 1 --2- 172.21.15.72:0/1985827735 >> v2:172.21.15.113:3300/0 conn(0x7fc89c155a50 0x7fc89c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc89bfff640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0b32a0 0x7fc89c0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.586+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286249177 0 0) 0x7fc89c0b5a10 con 0x7fc89c15b970 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc888003520 con 0x7fc89c15b970 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3446484031 0 0) 0x7fc888003520 con 0x7fc89c15b970 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0b32a0 msgr2=0x7fc89c0cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0b32a0 0x7fc89c0cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 >> v2:172.21.15.113:3300/0 conn(0x7fc89c155a50 msgr2=0x7fc89c0b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 --2- 172.21.15.72:0/1985827735 >> v2:172.21.15.113:3300/0 conn(0x7fc89c155a50 0x7fc89c0b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0b5fd0 con 0x7fc89c15b970 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fc88c001b70 con 0x7fc89c15b970 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 >> v1:172.21.15.72:6789/0 conn(0x7fc89c15b970 legacy=0x7fc89c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc888003970 0x7fc888003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8880047c0 0x7fc888004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0b32a0 0x7fc888005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b5cf0 con 0x7fc888003970 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88800b980 con 0x7fc8880047c0 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88800baf0 con 0x7fc89c0b32a0 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.587+0000 7fc89b7fe640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc888003970 0x7fc888003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.588+0000 7fc89affd640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8880047c0 0x7fc888004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.588+0000 7fc89b7fe640 1 -- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0b32a0 msgr2=0x7fc888005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.588+0000 7fc89b7fe640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0b32a0 0x7fc888005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.588+0000 7fc89b7fe640 1 -- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8880047c0 msgr2=0x7fc888004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.588+0000 7fc89b7fe640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8880047c0 0x7fc888004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.588+0000 7fc89b7fe640 1 -- 172.21.15.72:0/1985827735 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc888003520 con 0x7fc888003970 2023-12-12T09:52:25.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.589+0000 7fc89b7fe640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc888003970 0x7fc888003d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc884009ac0 tx=0x7fc8880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8990617da5a1bf6d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:25.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88402c6d0 con 0x7fc888003970 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88402c830 con 0x7fc888003970 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc89a7fc640 1 -- 172.21.15.72:0/1985827735 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88402cbc0 con 0x7fc888003970 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/1985827735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc888003970 msgr2=0x7fc888003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc8a1e9b640 1 --2- 172.21.15.72:0/1985827735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc888003970 0x7fc888003d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc884009ac0 tx=0x7fc8880057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc89bfff640 1 -- 172.21.15.72:0/1985827735 reap_dead start 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/1985827735 shutdown_connections 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/1985827735 >> 172.21.15.72:0/1985827735 conn(0x7fc89c0b6420 msgr2=0x7fc89c0b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.590+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/1985827735 shutdown_connections 2023-12-12T09:52:25.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.591+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/1985827735 wait complete. 2023-12-12T09:52:25.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.592+0000 7fc8a1e9b640 1 Processor -- start 2023-12-12T09:52:25.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.592+0000 7fc8a1e9b640 1 -- start start 2023-12-12T09:52:25.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.593+0000 7fc8a1e9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc888003970 0x7fc89c147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.593+0000 7fc8a1e9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0d0f00 0x7fc89c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.593+0000 7fc8a1e9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc89c06b700 0x7fc89c06bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc8a1e9b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0d18c0 con 0x7fc89c06b700 2023-12-12T09:52:25.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc8a1e9b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c06d100 con 0x7fc888003970 2023-12-12T09:52:25.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc8a1e9b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c06d590 con 0x7fc89c0d0f00 2023-12-12T09:52:25.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc89c06b700 0x7fc89c06bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc89c06b700 0x7fc89c06bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53606/0 (socket says 172.21.15.72:53606) 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89bfff640 1 -- 172.21.15.72:0/616071818 learned_addr learned my addr 172.21.15.72:0/616071818 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89b7fe640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc888003970 0x7fc89c147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89affd640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0d0f00 0x7fc89c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89bfff640 1 -- 172.21.15.72:0/616071818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0d0f00 msgr2=0x7fc89c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89bfff640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0d0f00 0x7fc89c0d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89bfff640 1 -- 172.21.15.72:0/616071818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc888003970 msgr2=0x7fc89c147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89bfff640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc888003970 0x7fc89c147900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.594+0000 7fc89bfff640 1 -- 172.21.15.72:0/616071818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0b57e0 con 0x7fc89c06b700 2023-12-12T09:52:25.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.595+0000 7fc89bfff640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc89c06b700 0x7fc89c06bb80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0d1f80 tx=0x7fc89400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:25.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.595+0000 7fc893fff640 1 -- 172.21.15.72:0/616071818 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400c680 con 0x7fc89c06b700 2023-12-12T09:52:25.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.595+0000 7fc893fff640 1 -- 172.21.15.72:0/616071818 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89400c7e0 con 0x7fc89c06b700 2023-12-12T09:52:25.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.595+0000 7fc893fff640 1 -- 172.21.15.72:0/616071818 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400cb90 con 0x7fc89c06b700 2023-12-12T09:52:25.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.595+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c147e40 con 0x7fc89c06b700 2023-12-12T09:52:25.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.595+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c06cab0 con 0x7fc89c06b700 2023-12-12T09:52:25.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.596+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c0cd5c0 con 0x7fc89c06b700 2023-12-12T09:52:25.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.597+0000 7fc893fff640 1 -- 172.21.15.72:0/616071818 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc89400ccf0 con 0x7fc89c06b700 2023-12-12T09:52:25.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.598+0000 7fc893fff640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc87c06ae40 0x7fc87c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:25.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.598+0000 7fc893fff640 1 -- 172.21.15.72:0/616071818 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc894089a40 con 0x7fc89c06b700 2023-12-12T09:52:25.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.598+0000 7fc89b7fe640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc87c06ae40 0x7fc87c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:25.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.599+0000 7fc89b7fe640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc87c06ae40 0x7fc87c06d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc884036000 tx=0x7fc884030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:25.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.604+0000 7fc893fff640 1 -- 172.21.15.72:0/616071818 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8940547f0 con 0x7fc89c06b700 2023-12-12T09:52:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.796+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc89c0b4650 con 0x7fc89c06b700 2023-12-12T09:52:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.797+0000 7fc893fff640 1 -- 172.21.15.72:0/616071818 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc894057e10 con 0x7fc89c06b700 2023-12-12T09:52:25.799 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:52:25.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.799+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc87c06ae40 msgr2=0x7fc87c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.799+0000 7fc8a1e9b640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc87c06ae40 0x7fc87c06d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc884036000 tx=0x7fc884030000 comp rx=0 tx=0).stop 2023-12-12T09:52:25.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.799+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc89c06b700 msgr2=0x7fc89c06bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:25.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.799+0000 7fc8a1e9b640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc89c06b700 0x7fc89c06bb80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0d1f80 tx=0x7fc89400ec00 comp rx=0 tx=0).stop 2023-12-12T09:52:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.799+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 shutdown_connections 2023-12-12T09:52:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.799+0000 7fc8a1e9b640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc87c06ae40 0x7fc87c06d300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.799+0000 7fc8a1e9b640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc89c06b700 0x7fc89c06bb80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.799+0000 7fc8a1e9b640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc89c0d0f00 0x7fc89c0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.800+0000 7fc8a1e9b640 1 --2- 172.21.15.72:0/616071818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc888003970 0x7fc89c147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.800+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 >> 172.21.15.72:0/616071818 conn(0x7fc89c0b6420 msgr2=0x7fc89c0b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.800+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 shutdown_connections 2023-12-12T09:52:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:25.800+0000 7fc8a1e9b640 1 -- 172.21.15.72:0/616071818 wait complete. 2023-12-12T09:52:25.821 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:52:26.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.036+0000 7fab3e325640 1 Processor -- start 2023-12-12T09:52:26.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.036+0000 7fab3e325640 1 -- start start 2023-12-12T09:52:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.038+0000 7fab3e325640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab3815d630 0x7fab3806be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.038+0000 7fab3e325640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fab3806b940 0x7fab3806c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.038+0000 7fab3e325640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab380b1780 con 0x7fab3815d250 2023-12-12T09:52:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.038+0000 7fab3e325640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab380b1a60 con 0x7fab3806b940 2023-12-12T09:52:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.038+0000 7fab3e325640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab380b1d40 con 0x7fab3815d630 2023-12-12T09:52:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.038+0000 7fab37fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fab3815d250 0x7fab380b0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42064/0 (socket says 172.21.15.72:42064) 2023-12-12T09:52:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.038+0000 7fab37fff640 1 -- 172.21.15.72:0/3835987040 learned_addr learned my addr 172.21.15.72:0/3835987040 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.038+0000 7fab3c8c2640 1 --2- 172.21.15.72:0/3835987040 >> v2:172.21.15.72:3300/0 conn(0x7fab3806b940 0x7fab3806c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.039+0000 7fab377fe640 1 --2- 172.21.15.72:0/3835987040 >> v2:172.21.15.113:3300/0 conn(0x7fab3815d630 0x7fab3806be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.039+0000 7fab3c8c2640 1 -- 172.21.15.72:0/3835987040 >> v2:172.21.15.113:3300/0 conn(0x7fab3815d630 msgr2=0x7fab3806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.039+0000 7fab36ffd640 1 -- 172.21.15.72:0/3835987040 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850124030 0 0) 0x7fab380b1780 con 0x7fab3815d250 2023-12-12T09:52:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.039+0000 7fab3c8c2640 1 --2- 172.21.15.72:0/3835987040 >> v2:172.21.15.113:3300/0 conn(0x7fab3815d630 0x7fab3806be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.039+0000 7fab3c8c2640 1 -- 172.21.15.72:0/3835987040 >> v1:172.21.15.72:6789/0 conn(0x7fab3815d250 legacy=0x7fab380b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.039+0000 7fab3c8c2640 1 -- 172.21.15.72:0/3835987040 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380b2020 con 0x7fab3806b940 2023-12-12T09:52:26.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.040+0000 7fab3c8c2640 1 --2- 172.21.15.72:0/3835987040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab3806c3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab380713d0 tx=0x7fab3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c131014eebd8e2d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.040+0000 7fab36ffd640 1 -- 172.21.15.72:0/3835987040 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3002cc80 con 0x7fab3806b940 2023-12-12T09:52:26.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.040+0000 7fab36ffd640 1 -- 172.21.15.72:0/3835987040 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3002cde0 con 0x7fab3806b940 2023-12-12T09:52:26.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.040+0000 7fab36ffd640 1 -- 172.21.15.72:0/3835987040 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30034660 con 0x7fab3806b940 2023-12-12T09:52:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.041+0000 7fab3e325640 1 -- 172.21.15.72:0/3835987040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 msgr2=0x7fab3806c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.041+0000 7fab3e325640 1 --2- 172.21.15.72:0/3835987040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab3806c3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab380713d0 tx=0x7fab3002bed0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.041+0000 7fab3e325640 1 -- 172.21.15.72:0/3835987040 shutdown_connections 2023-12-12T09:52:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.041+0000 7fab3e325640 1 --2- 172.21.15.72:0/3835987040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab3806c3c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.041+0000 7fab3e325640 1 --2- 172.21.15.72:0/3835987040 >> v2:172.21.15.113:3300/0 conn(0x7fab3815d630 0x7fab3806be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.041+0000 7fab3e325640 1 -- 172.21.15.72:0/3835987040 >> 172.21.15.72:0/3835987040 conn(0x7fab380d0e90 msgr2=0x7fab380d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.041+0000 7fab3e325640 1 -- 172.21.15.72:0/3835987040 shutdown_connections 2023-12-12T09:52:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.041+0000 7fab3e325640 1 -- 172.21.15.72:0/3835987040 wait complete. 2023-12-12T09:52:26.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.043+0000 7fab3e325640 1 Processor -- start 2023-12-12T09:52:26.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.043+0000 7fab3e325640 1 -- start start 2023-12-12T09:52:26.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab3e325640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab380c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab3e325640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3815d250 0x7fab38157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab3e325640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab3815d630 0x7fab380c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab3e325640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38157db0 con 0x7fab3806b940 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab3e325640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab380c5ef0 con 0x7fab3815d250 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab3e325640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab380c61d0 con 0x7fab3815d630 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab37fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab380c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab37fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab380c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53622/0 (socket says 172.21.15.72:53622) 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.044+0000 7fab37fff640 1 -- 172.21.15.72:0/2112078661 learned_addr learned my addr 172.21.15.72:0/2112078661 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab377fe640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3815d250 0x7fab38157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab3c8c2640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab3815d630 0x7fab380c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab37fff640 1 -- 172.21.15.72:0/2112078661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab3815d630 msgr2=0x7fab380c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab37fff640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab3815d630 0x7fab380c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab37fff640 1 -- 172.21.15.72:0/2112078661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3815d250 msgr2=0x7fab38157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab37fff640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3815d250 0x7fab38157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab37fff640 1 -- 172.21.15.72:0/2112078661 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009a70 con 0x7fab3806b940 2023-12-12T09:52:26.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab37fff640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab380c7440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab2800c4d0 tx=0x7fab2800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab34ff9640 1 -- 172.21.15.72:0/2112078661 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab280074e0 con 0x7fab3806b940 2023-12-12T09:52:26.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab34ff9640 1 -- 172.21.15.72:0/2112078661 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28007660 con 0x7fab3806b940 2023-12-12T09:52:26.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab34ff9640 1 -- 172.21.15.72:0/2112078661 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28004a20 con 0x7fab3806b940 2023-12-12T09:52:26.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab380c64b0 con 0x7fab3806b940 2023-12-12T09:52:26.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.045+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab380c5b70 con 0x7fab3806b940 2023-12-12T09:52:26.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.047+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab380673f0 con 0x7fab3806b940 2023-12-12T09:52:26.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.048+0000 7fab34ff9640 1 -- 172.21.15.72:0/2112078661 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab28004b80 con 0x7fab3806b940 2023-12-12T09:52:26.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.048+0000 7fab34ff9640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.048+0000 7fab34ff9640 1 -- 172.21.15.72:0/2112078661 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fab28088af0 con 0x7fab3806b940 2023-12-12T09:52:26.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.049+0000 7fab377fe640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.049+0000 7fab377fe640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fab2406ae40 0x7fab2406d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab2c006fd0 tx=0x7fab2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.053+0000 7fab34ff9640 1 -- 172.21.15.72:0/2112078661 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab280538b0 con 0x7fab3806b940 2023-12-12T09:52:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.241+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab380b3ce0 con 0x7fab3806b940 2023-12-12T09:52:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.242+0000 7fab34ff9640 1 -- 172.21.15.72:0/2112078661 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab28018020 con 0x7fab3806b940 2023-12-12T09:52:26.243 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:52:26.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fab2406ae40 msgr2=0x7fab2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fab2406ae40 0x7fab2406d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab2c006fd0 tx=0x7fab2c008040 comp rx=0 tx=0).stop 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 msgr2=0x7fab380c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab380c7440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab2800c4d0 tx=0x7fab2800c990 comp rx=0 tx=0).stop 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 shutdown_connections 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fab2406ae40 0x7fab2406d300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab3815d630 0x7fab380c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3815d250 0x7fab38157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 --2- 172.21.15.72:0/2112078661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab3806b940 0x7fab380c7440 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.244+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 >> 172.21.15.72:0/2112078661 conn(0x7fab380d0e90 msgr2=0x7fab3806be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.245+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 shutdown_connections 2023-12-12T09:52:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.245+0000 7fab3e325640 1 -- 172.21.15.72:0/2112078661 wait complete. 2023-12-12T09:52:26.265 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:52:26.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.477+0000 7f2dead89640 1 Processor -- start 2023-12-12T09:52:26.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.478+0000 7f2dead89640 1 -- start start 2023-12-12T09:52:26.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.479+0000 7f2dead89640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2de4071e80 0x7f2de406d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.479+0000 7f2dead89640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de4165bd0 con 0x7f2de4166060 2023-12-12T09:52:26.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.479+0000 7f2dead89640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de40b4c80 con 0x7f2de41588e0 2023-12-12T09:52:26.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.479+0000 7f2dead89640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de40b4f60 con 0x7f2de4071e80 2023-12-12T09:52:26.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.479+0000 7f2de9326640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f2de4166060 0x7f2de4161df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42074/0 (socket says 172.21.15.72:42074) 2023-12-12T09:52:26.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2de9326640 1 -- 172.21.15.72:0/376823426 learned_addr learned my addr 172.21.15.72:0/376823426 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddbfff640 1 --2- 172.21.15.72:0/376823426 >> v2:172.21.15.113:3300/0 conn(0x7f2de4071e80 0x7f2de406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622903576 0 0) 0x7f2de4165bd0 con 0x7f2de4166060 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dcc003520 con 0x7f2de4166060 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758025680 0 0) 0x7f2dcc003520 con 0x7f2de4166060 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 >> v2:172.21.15.113:3300/0 conn(0x7f2de4071e80 msgr2=0x7f2de406d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/376823426 >> v2:172.21.15.113:3300/0 conn(0x7f2de4071e80 0x7f2de406d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 >> v1:172.21.15.113:6789/0 conn(0x7f2de41588e0 legacy=0x7f2de40b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4165580 con 0x7f2de4166060 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f2ddc0019f0 con 0x7f2de4166060 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.480+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 >> v1:172.21.15.72:6789/0 conn(0x7f2de4166060 legacy=0x7f2de4161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dcc0039c0 0x7f2dcc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0047c0 0x7f2dcc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0051d0 0x7f2dcc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40b4f60 con 0x7f2dcc0039c0 2023-12-12T09:52:26.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc00c120 con 0x7f2dcc0047c0 2023-12-12T09:52:26.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc00c2e0 con 0x7f2dcc0051d0 2023-12-12T09:52:26.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2de8b25640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dcc0039c0 0x7f2dcc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2ddbfff640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0047c0 0x7f2dcc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2de8b25640 1 -- 172.21.15.72:0/376823426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0051d0 msgr2=0x7f2dcc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:26.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2de8b25640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0051d0 0x7f2dcc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2de8b25640 1 -- 172.21.15.72:0/376823426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0047c0 msgr2=0x7f2dcc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2de8b25640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0047c0 0x7f2dcc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2de8b25640 1 -- 172.21.15.72:0/376823426 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc003520 con 0x7f2dcc0039c0 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2ddbfff640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0047c0 0x7f2dcc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.481+0000 7f2de9326640 1 -- 172.21.15.72:0/376823426 reap_dead start 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.482+0000 7f2de8b25640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dcc0039c0 0x7f2dcc003de0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2de0001720 tx=0x7f2dcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93936d199973f8ca server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00052d0 con 0x7f2dcc0039c0 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de002e050 con 0x7f2dcc0039c0 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/376823426 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0003830 con 0x7f2dcc0039c0 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2dead89640 1 -- 172.21.15.72:0/376823426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dcc0039c0 msgr2=0x7f2dcc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2dead89640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dcc0039c0 0x7f2dcc003de0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2de0001720 tx=0x7f2dcc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2dead89640 1 -- 172.21.15.72:0/376823426 shutdown_connections 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2dead89640 1 --2- 172.21.15.72:0/376823426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dcc0039c0 0x7f2dcc003de0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2dead89640 1 -- 172.21.15.72:0/376823426 >> 172.21.15.72:0/376823426 conn(0x7f2de40cb850 msgr2=0x7f2de40cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:26.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2dead89640 1 -- 172.21.15.72:0/376823426 shutdown_connections 2023-12-12T09:52:26.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.483+0000 7f2dead89640 1 -- 172.21.15.72:0/376823426 wait complete. 2023-12-12T09:52:26.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.485+0000 7f2dead89640 1 Processor -- start 2023-12-12T09:52:26.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.485+0000 7f2dead89640 1 -- start start 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.486+0000 7f2dead89640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0039c0 0x7f2de4155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.486+0000 7f2dead89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de4162270 0x7f2de40aa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.486+0000 7f2dead89640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de41626f0 0x7f2de40aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.486+0000 7f2dead89640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4161190 con 0x7f2de41626f0 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.486+0000 7f2dead89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de41562b0 con 0x7f2de4162270 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.486+0000 7f2dead89640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de41605c0 con 0x7f2dcc0039c0 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de41626f0 0x7f2de40aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de41626f0 0x7f2de40aa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53652/0 (socket says 172.21.15.72:53652) 2023-12-12T09:52:26.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 -- 172.21.15.72:0/4291472905 learned_addr learned my addr 172.21.15.72:0/4291472905 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de8b25640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0039c0 0x7f2de4155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2ddbfff640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de4162270 0x7f2de40aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 -- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0039c0 msgr2=0x7f2de4155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0039c0 0x7f2de4155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 -- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de4162270 msgr2=0x7f2de40aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de4162270 0x7f2de40aa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 -- 172.21.15.72:0/4291472905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4160860 con 0x7f2de41626f0 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de8b25640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0039c0 0x7f2de4155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:26.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.487+0000 7f2de9326640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de41626f0 0x7f2de40aa660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2de40cec30 tx=0x7f2ddc00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.488+0000 7f2dd97fa640 1 -- 172.21.15.72:0/4291472905 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc004d60 con 0x7f2de41626f0 2023-12-12T09:52:26.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.488+0000 7f2dd97fa640 1 -- 172.21.15.72:0/4291472905 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc00a610 con 0x7f2de41626f0 2023-12-12T09:52:26.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.488+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de4160b40 con 0x7f2de41626f0 2023-12-12T09:52:26.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.488+0000 7f2dd97fa640 1 -- 172.21.15.72:0/4291472905 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00a9c0 con 0x7f2de41626f0 2023-12-12T09:52:26.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.488+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de40ad440 con 0x7f2de41626f0 2023-12-12T09:52:26.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.489+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de40ad5d0 con 0x7f2de41626f0 2023-12-12T09:52:26.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.490+0000 7f2dd97fa640 1 -- 172.21.15.72:0/4291472905 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ddc00ab20 con 0x7f2de41626f0 2023-12-12T09:52:26.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.491+0000 7f2dd97fa640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dd006ae40 0x7f2dd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.491+0000 7f2dd97fa640 1 -- 172.21.15.72:0/4291472905 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2ddc0894a0 con 0x7f2de41626f0 2023-12-12T09:52:26.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.491+0000 7f2de8b25640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dd006ae40 0x7f2dd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.492+0000 7f2de8b25640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dd006ae40 0x7f2dd006d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2de0033000 tx=0x7f2de0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.495+0000 7f2dd97fa640 1 -- 172.21.15.72:0/4291472905 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ddc054250 con 0x7f2de41626f0 2023-12-12T09:52:26.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.685+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2de40745d0 con 0x7f2de41626f0 2023-12-12T09:52:26.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.685+0000 7f2dd97fa640 1 -- 172.21.15.72:0/4291472905 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2ddc057870 con 0x7f2de41626f0 2023-12-12T09:52:26.687 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dd006ae40 msgr2=0x7f2dd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dd006ae40 0x7f2dd006d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2de0033000 tx=0x7f2de0009030 comp rx=0 tx=0).stop 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de41626f0 msgr2=0x7f2de40aa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de41626f0 0x7f2de40aa660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2de40cec30 tx=0x7f2ddc00cb80 comp rx=0 tx=0).stop 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 shutdown_connections 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dd006ae40 0x7f2dd006d300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de41626f0 0x7f2de40aa660 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de4162270 0x7f2de40aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 --2- 172.21.15.72:0/4291472905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc0039c0 0x7f2de4155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 >> 172.21.15.72:0/4291472905 conn(0x7f2de40cb850 msgr2=0x7f2de41568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:26.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 shutdown_connections 2023-12-12T09:52:26.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.689+0000 7f2dead89640 1 -- 172.21.15.72:0/4291472905 wait complete. 2023-12-12T09:52:26.710 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:52:26.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.924+0000 7f1570290640 1 Processor -- start 2023-12-12T09:52:26.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.924+0000 7f1570290640 1 -- start start 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f1570290640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1568157a60 0x7f156806c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f1570290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156806de20 0x7f156815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f1570290640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156806ca40 con 0x7f156815e0e0 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f1570290640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156815e550 con 0x7f1568157a60 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f1570290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1568071cd0 con 0x7f156806de20 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f156e02c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1568157a60 0x7f156806c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f156e02c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1568157a60 0x7f156806c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53664/0 (socket says 172.21.15.72:53664) 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f156e02c640 1 -- 172.21.15.72:0/1011782858 learned_addr learned my addr 172.21.15.72:0/1011782858 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:26.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.926+0000 7f156d82b640 1 --2- 172.21.15.72:0/1011782858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156806de20 0x7f156815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.927+0000 7f156e02c640 1 -- 172.21.15.72:0/1011782858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156806de20 msgr2=0x7f156815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:26.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.927+0000 7f156e02c640 1 --2- 172.21.15.72:0/1011782858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156806de20 0x7f156815dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.927+0000 7f156e02c640 1 -- 172.21.15.72:0/1011782858 >> v1:172.21.15.113:6789/0 conn(0x7f156815e0e0 legacy=0x7f156815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:26.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.927+0000 7f156e02c640 1 -- 172.21.15.72:0/1011782858 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568070f10 con 0x7f1568157a60 2023-12-12T09:52:26.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.928+0000 7f156e02c640 1 --2- 172.21.15.72:0/1011782858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1568157a60 0x7f156806c410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f155800d150 tx=0x7f155802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc92ce66e224484 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.928+0000 7f156d02a640 1 -- 172.21.15.72:0/1011782858 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155802e9e0 con 0x7f1568157a60 2023-12-12T09:52:26.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.928+0000 7f156d02a640 1 -- 172.21.15.72:0/1011782858 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155802eb40 con 0x7f1568157a60 2023-12-12T09:52:26.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.928+0000 7f156d02a640 1 -- 172.21.15.72:0/1011782858 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558009600 con 0x7f1568157a60 2023-12-12T09:52:26.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.929+0000 7f1570290640 1 -- 172.21.15.72:0/1011782858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1568157a60 msgr2=0x7f156806c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.929+0000 7f1570290640 1 --2- 172.21.15.72:0/1011782858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1568157a60 0x7f156806c410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f155800d150 tx=0x7f155802e040 comp rx=0 tx=0).stop 2023-12-12T09:52:26.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.929+0000 7f1570290640 1 -- 172.21.15.72:0/1011782858 shutdown_connections 2023-12-12T09:52:26.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.929+0000 7f1570290640 1 --2- 172.21.15.72:0/1011782858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156806de20 0x7f156815dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.929+0000 7f1570290640 1 --2- 172.21.15.72:0/1011782858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1568157a60 0x7f156806c410 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.929+0000 7f1570290640 1 -- 172.21.15.72:0/1011782858 >> 172.21.15.72:0/1011782858 conn(0x7f15680b0a90 msgr2=0x7f15680b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:26.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.929+0000 7f1570290640 1 -- 172.21.15.72:0/1011782858 shutdown_connections 2023-12-12T09:52:26.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.929+0000 7f1570290640 1 -- 172.21.15.72:0/1011782858 wait complete. 2023-12-12T09:52:26.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.931+0000 7f1570290640 1 Processor -- start 2023-12-12T09:52:26.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.931+0000 7f1570290640 1 -- start start 2023-12-12T09:52:26.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f1570290640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f156806de20 0x7f156814aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f1570290640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1568157a60 0x7f15680b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f1570290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156815e0e0 0x7f1568152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f1570290640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1568158210 con 0x7f156806de20 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f1570290640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1568157ee0 con 0x7f1568157a60 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f1570290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15681ee830 con 0x7f156815e0e0 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f156e02c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f156806de20 0x7f156814aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f156e02c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f156806de20 0x7f156814aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53678/0 (socket says 172.21.15.72:53678) 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.932+0000 7f156e02c640 1 -- 172.21.15.72:0/3843945750 learned_addr learned my addr 172.21.15.72:0/3843945750 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:26.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f156d82b640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1568157a60 0x7f15680b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f156e82d640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156815e0e0 0x7f1568152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f156e02c640 1 -- 172.21.15.72:0/3843945750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156815e0e0 msgr2=0x7f1568152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f156e02c640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156815e0e0 0x7f1568152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f156e02c640 1 -- 172.21.15.72:0/3843945750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1568157a60 msgr2=0x7f15680b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f156e02c640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1568157a60 0x7f15680b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f156e02c640 1 -- 172.21.15.72:0/3843945750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155800bd40 con 0x7f156806de20 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f156e02c640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f156806de20 0x7f156814aab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1558000f80 tx=0x7f1558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f1566ffd640 1 -- 172.21.15.72:0/3843945750 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155802e9e0 con 0x7f156806de20 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f1566ffd640 1 -- 172.21.15.72:0/3843945750 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155800b720 con 0x7f156806de20 2023-12-12T09:52:26.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15680b4960 con 0x7f156806de20 2023-12-12T09:52:26.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.933+0000 7f1566ffd640 1 -- 172.21.15.72:0/3843945750 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580408a0 con 0x7f156806de20 2023-12-12T09:52:26.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.934+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15680b4f20 con 0x7f156806de20 2023-12-12T09:52:26.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.935+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15680745d0 con 0x7f156806de20 2023-12-12T09:52:26.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.936+0000 7f1566ffd640 1 -- 172.21.15.72:0/3843945750 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1558040a00 con 0x7f156806de20 2023-12-12T09:52:26.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.937+0000 7f1566ffd640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f155406ae40 0x7f155406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:26.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.937+0000 7f1566ffd640 1 -- 172.21.15.72:0/3843945750 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f15580ae8c0 con 0x7f156806de20 2023-12-12T09:52:26.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.937+0000 7f156d82b640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f155406ae40 0x7f155406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:26.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.938+0000 7f156d82b640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f155406ae40 0x7f155406d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15680b34e0 tx=0x7f155c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:26.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:26.942+0000 7f1566ffd640 1 -- 172.21.15.72:0/3843945750 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1558079740 con 0x7f156806de20 2023-12-12T09:52:27.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.142+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f156806ce20 con 0x7f156806de20 2023-12-12T09:52:27.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.143+0000 7f1566ffd640 1 -- 172.21.15.72:0/3843945750 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1558040cb0 con 0x7f156806de20 2023-12-12T09:52:27.145 INFO:teuthology.orchestra.run.smithi072.stderr:listed 0 entries 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f155406ae40 msgr2=0x7f155406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f155406ae40 0x7f155406d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15680b34e0 tx=0x7f155c008040 comp rx=0 tx=0).stop 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f156806de20 msgr2=0x7f156814aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f156806de20 0x7f156814aab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1558000f80 tx=0x7f1558004a40 comp rx=0 tx=0).stop 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 shutdown_connections 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f155406ae40 0x7f155406d300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f156815e0e0 0x7f1568152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1568157a60 0x7f15680b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:27.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 --2- 172.21.15.72:0/3843945750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f156806de20 0x7f156814aab0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:27.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 >> 172.21.15.72:0/3843945750 conn(0x7f15680b0a90 msgr2=0x7f15680aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:27.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 shutdown_connections 2023-12-12T09:52:27.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:27.146+0000 7f1570290640 1 -- 172.21.15.72:0/3843945750 wait complete. 2023-12-12T09:52:27.167 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi072.front.sepia.ceph.com... 2023-12-12T09:52:27.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:27.167 DEBUG:teuthology.orchestra.run.smithi072:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:52:27.251 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi072.stderr:ceph-fuse[95124]: fuse finished with error 0 and tester_r 0 2023-12-12T09:52:27.256 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:52:32.840 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:52:32.849 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:52:33.357 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:52:33.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:33.358 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-12-12T09:52:33.425 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:33.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.617+0000 7f1c6887c640 1 Processor -- start 2023-12-12T09:52:33.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.617+0000 7f1c6887c640 1 -- start start 2023-12-12T09:52:33.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.619+0000 7f1c6887c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1c640d3dd0 0x7f1c640cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:33.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.619+0000 7f1c6887c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c640cdc40 con 0x7f1c640cfb80 2023-12-12T09:52:33.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.619+0000 7f1c6887c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c640d1e60 con 0x7f1c6415d480 2023-12-12T09:52:33.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.619+0000 7f1c6887c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c640cde70 con 0x7f1c640d3dd0 2023-12-12T09:52:33.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.619+0000 7f1c5bfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1c640d3dd0 0x7f1c640cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:33.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.619+0000 7f1c5bfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1c640d3dd0 0x7f1c640cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53686/0 (socket says 172.21.15.72:53686) 2023-12-12T09:52:33.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.619+0000 7f1c5bfff640 1 -- 172.21.15.72:0/1443177254 learned_addr learned my addr 172.21.15.72:0/1443177254 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:33.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.620+0000 7f1c5bfff640 1 -- 172.21.15.72:0/1443177254 >> v1:172.21.15.113:6789/0 conn(0x7f1c6415d480 legacy=0x7f1c640d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:33.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.620+0000 7f1c5bfff640 1 -- 172.21.15.72:0/1443177254 >> v1:172.21.15.72:6789/0 conn(0x7f1c640cfb80 legacy=0x7f1c640c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:33.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.620+0000 7f1c5bfff640 1 -- 172.21.15.72:0/1443177254 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c640ce150 con 0x7f1c640d3dd0 2023-12-12T09:52:33.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.620+0000 7f1c61d9b640 1 -- 172.21.15.72:0/1443177254 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241181840 0 0) 0x7f1c640cdc40 con 0x7f1c640cfb80 2023-12-12T09:52:33.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.621+0000 7f1c5bfff640 1 --2- 172.21.15.72:0/1443177254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640d3dd0 0x7f1c640cec30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009870 tx=0x7f1c4c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9fdcbc482d044d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:33.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.621+0000 7f1c61d9b640 1 -- 172.21.15.72:0/1443177254 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c02f840 con 0x7f1c640d3dd0 2023-12-12T09:52:33.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.621+0000 7f1c61d9b640 1 -- 172.21.15.72:0/1443177254 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c02f9a0 con 0x7f1c640d3dd0 2023-12-12T09:52:33.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.621+0000 7f1c61d9b640 1 -- 172.21.15.72:0/1443177254 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c02fd30 con 0x7f1c640d3dd0 2023-12-12T09:52:33.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.622+0000 7f1c6887c640 1 -- 172.21.15.72:0/1443177254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640d3dd0 msgr2=0x7f1c640cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:33.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.622+0000 7f1c6887c640 1 --2- 172.21.15.72:0/1443177254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640d3dd0 0x7f1c640cec30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009870 tx=0x7f1c4c02ba20 comp rx=0 tx=0).stop 2023-12-12T09:52:33.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.622+0000 7f1c6887c640 1 -- 172.21.15.72:0/1443177254 shutdown_connections 2023-12-12T09:52:33.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.622+0000 7f1c6887c640 1 --2- 172.21.15.72:0/1443177254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640d3dd0 0x7f1c640cec30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:33.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.622+0000 7f1c6887c640 1 -- 172.21.15.72:0/1443177254 >> 172.21.15.72:0/1443177254 conn(0x7f1c6409c4b0 msgr2=0x7f1c640d3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:33.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.622+0000 7f1c6887c640 1 -- 172.21.15.72:0/1443177254 shutdown_connections 2023-12-12T09:52:33.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.622+0000 7f1c6887c640 1 -- 172.21.15.72:0/1443177254 wait complete. 2023-12-12T09:52:33.626 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.624+0000 7f1c6887c640 1 Processor -- start 2023-12-12T09:52:33.626 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.624+0000 7f1c6887c640 1 -- start start 2023-12-12T09:52:33.627 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6887c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640cfb80 0x7f1c640ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6887c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c640d3dd0 0x7f1c640b6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6887c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c6415d480 0x7f1c640b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6887c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640cd1d0 con 0x7f1c640cfb80 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6887c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c64160560 con 0x7f1c6415d480 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6887c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640caf70 con 0x7f1c640d3dd0 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640cfb80 0x7f1c640ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640cfb80 0x7f1c640ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53692/0 (socket says 172.21.15.72:53692) 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.625+0000 7f1c6259c640 1 -- 172.21.15.72:0/1559926408 learned_addr learned my addr 172.21.15.72:0/1559926408 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c62d9d640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c6415d480 0x7f1c640b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c5bfff640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c640d3dd0 0x7f1c640b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c6259c640 1 -- 172.21.15.72:0/1559926408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c640d3dd0 msgr2=0x7f1c640b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:33.628 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c6259c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c640d3dd0 0x7f1c640b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:33.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c6259c640 1 -- 172.21.15.72:0/1559926408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c6415d480 msgr2=0x7f1c640b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:33.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c6259c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c6415d480 0x7f1c640b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:33.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c6259c640 1 -- 172.21.15.72:0/1559926408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c0094d0 con 0x7f1c640cfb80 2023-12-12T09:52:33.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c6259c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640cfb80 0x7f1c640ccc90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c450 tx=0x7f1c5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:33.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c5affd640 1 -- 172.21.15.72:0/1559926408 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540052d0 con 0x7f1c640cfb80 2023-12-12T09:52:33.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c5affd640 1 -- 172.21.15.72:0/1559926408 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c54005430 con 0x7f1c640cfb80 2023-12-12T09:52:33.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6415fcc0 con 0x7f1c640cfb80 2023-12-12T09:52:33.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.626+0000 7f1c5affd640 1 -- 172.21.15.72:0/1559926408 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54012870 con 0x7f1c640cfb80 2023-12-12T09:52:33.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.627+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c64160280 con 0x7f1c640cfb80 2023-12-12T09:52:33.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.628+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c640745d0 con 0x7f1c640cfb80 2023-12-12T09:52:33.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.629+0000 7f1c5affd640 1 -- 172.21.15.72:0/1559926408 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c540129d0 con 0x7f1c640cfb80 2023-12-12T09:52:33.631 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.630+0000 7f1c5affd640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1c4406ae40 0x7f1c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:33.631 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.630+0000 7f1c5affd640 1 -- 172.21.15.72:0/1559926408 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1c540893b0 con 0x7f1c640cfb80 2023-12-12T09:52:33.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.630+0000 7f1c5bfff640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1c4406ae40 0x7f1c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:33.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.631+0000 7f1c5bfff640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1c4406ae40 0x7f1c4406d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c036000 tx=0x7f1c4c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:33.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.635+0000 7f1c5affd640 1 -- 172.21.15.72:0/1559926408 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c54054200 con 0x7f1c640cfb80 2023-12-12T09:52:33.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.828+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c640cf960 con 0x7f1c640cfb80 2023-12-12T09:52:33.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.829+0000 7f1c5affd640 1 -- 172.21.15.72:0/1559926408 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f1c54057820 con 0x7f1c640cfb80 2023-12-12T09:52:33.831 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:33.832 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1c4406ae40 msgr2=0x7f1c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1c4406ae40 0x7f1c4406d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c036000 tx=0x7f1c4c008030 comp rx=0 tx=0).stop 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640cfb80 msgr2=0x7f1c640ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640cfb80 0x7f1c640ccc90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c450 tx=0x7f1c5400c910 comp rx=0 tx=0).stop 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 shutdown_connections 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1c4406ae40 0x7f1c4406d300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c6415d480 0x7f1c640b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c640d3dd0 0x7f1c640b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:33.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 --2- 172.21.15.72:0/1559926408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c640cfb80 0x7f1c640ccc90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:33.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 >> 172.21.15.72:0/1559926408 conn(0x7f1c6409c4b0 msgr2=0x7f1c640c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:33.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 shutdown_connections 2023-12-12T09:52:33.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:33.832+0000 7f1c6887c640 1 -- 172.21.15.72:0/1559926408 wait complete. 2023-12-12T09:52:33.852 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:34.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.040+0000 7f71a76cf640 1 Processor -- start 2023-12-12T09:52:34.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.040+0000 7f71a76cf640 1 -- start start 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a76cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 0x7f71a00d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a76cf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71a00d0230 0x7f71a015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a76cf640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a015dd60 con 0x7f71a00d1220 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a76cf640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a0160d90 con 0x7f71a00d0230 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a76cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015e040 con 0x7f71a01609b0 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a4c6a640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f71a00d1220 0x7f71a00cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42102/0 (socket says 172.21.15.72:42102) 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a4c6a640 1 -- 172.21.15.72:0/3636368237 learned_addr learned my addr 172.21.15.72:0/3636368237 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a5c6c640 1 --2- 172.21.15.72:0/3636368237 >> v2:172.21.15.113:3300/0 conn(0x7f71a00d0230 0x7f71a015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f71a546b640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 0x7f71a00d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931271606 0 0) 0x7f71a015dd60 con 0x7f71a00d1220 2023-12-12T09:52:34.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.042+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f718c003520 con 0x7f71a00d1220 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2567821085 0 0) 0x7f718c003520 con 0x7f71a00d1220 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 msgr2=0x7f71a00d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 0x7f71a00d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 >> v2:172.21.15.113:3300/0 conn(0x7f71a00d0230 msgr2=0x7f71a015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 --2- 172.21.15.72:0/3636368237 >> v2:172.21.15.113:3300/0 conn(0x7f71a00d0230 0x7f71a015cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a015e320 con 0x7f71a00d1220 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f7190001670 con 0x7f71a00d1220 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f71a546b640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 0x7f71a00d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 >> v1:172.21.15.72:6789/0 conn(0x7f71a00d1220 legacy=0x7f71a00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f718c003970 0x7f718c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c0047c0 0x7f718c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 0x7f718c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0160d90 con 0x7f718c003970 2023-12-12T09:52:34.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015e040 con 0x7f718c0047c0 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c00b980 con 0x7f71a01609b0 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.043+0000 7f71a546b640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f718c003970 0x7f718c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.044+0000 7f71a4c6a640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c0047c0 0x7f718c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.044+0000 7f71a5c6c640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 0x7f718c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.044+0000 7f71a546b640 1 -- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 msgr2=0x7f718c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.044+0000 7f71a546b640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a01609b0 0x7f718c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.044+0000 7f71a546b640 1 -- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c0047c0 msgr2=0x7f718c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.044+0000 7f71a546b640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c0047c0 0x7f718c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.044+0000 7f71a546b640 1 -- 172.21.15.72:0/3636368237 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c003520 con 0x7f718c003970 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.044+0000 7f71a4c6a640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f718c0047c0 0x7f718c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.045+0000 7f71a546b640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f718c003970 0x7f718c003d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f719c0098d0 tx=0x7f718c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55acddc9b977d8df server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.046+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c004230 con 0x7f718c003970 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.046+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c004390 con 0x7f718c003970 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.046+0000 7f7197fff640 1 -- 172.21.15.72:0/3636368237 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c003830 con 0x7f718c003970 2023-12-12T09:52:34.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.046+0000 7f71a76cf640 1 -- 172.21.15.72:0/3636368237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f718c003970 msgr2=0x7f718c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.046+0000 7f71a76cf640 1 --2- 172.21.15.72:0/3636368237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f718c003970 0x7f718c003d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f719c0098d0 tx=0x7f718c0057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.046+0000 7f71a5c6c640 1 -- 172.21.15.72:0/3636368237 reap_dead start 2023-12-12T09:52:34.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.047+0000 7f71a76cf640 1 -- 172.21.15.72:0/3636368237 shutdown_connections 2023-12-12T09:52:34.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.047+0000 7f71a76cf640 1 -- 172.21.15.72:0/3636368237 >> 172.21.15.72:0/3636368237 conn(0x7f71a009c450 msgr2=0x7f71a00d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:34.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.047+0000 7f71a76cf640 1 -- 172.21.15.72:0/3636368237 shutdown_connections 2023-12-12T09:52:34.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.047+0000 7f71a76cf640 1 -- 172.21.15.72:0/3636368237 wait complete. 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.048+0000 7f71a76cf640 1 Processor -- start 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.049+0000 7f71a76cf640 1 -- start start 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.049+0000 7f71a76cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f718c003970 0x7f71a014c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.049+0000 7f71a76cf640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a00afe90 0x7f71a00b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.049+0000 7f71a76cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a00b6d80 0x7f71a00b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.049+0000 7f71a76cf640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00b7770 con 0x7f71a00afe90 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.049+0000 7f71a76cf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a014f2e0 con 0x7f71a00b6d80 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.049+0000 7f71a76cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a014f5c0 con 0x7f718c003970 2023-12-12T09:52:34.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a00afe90 0x7f71a00b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a00afe90 0x7f71a00b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53716/0 (socket says 172.21.15.72:53716) 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 -- 172.21.15.72:0/2777399386 learned_addr learned my addr 172.21.15.72:0/2777399386 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a546b640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f718c003970 0x7f71a014c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a5c6c640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a00b6d80 0x7f71a00b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 -- 172.21.15.72:0/2777399386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f718c003970 msgr2=0x7f71a014c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f718c003970 0x7f71a014c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 -- 172.21.15.72:0/2777399386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a00b6d80 msgr2=0x7f71a00b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a00b6d80 0x7f71a00b7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 -- 172.21.15.72:0/2777399386 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a014f8a0 con 0x7f71a00afe90 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.050+0000 7f71a4c6a640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a00afe90 0x7f71a00b0310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7190000ff0 tx=0x7f719000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.051+0000 7f7195ffb640 1 -- 172.21.15.72:0/2777399386 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000c870 con 0x7f71a00afe90 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.051+0000 7f7195ffb640 1 -- 172.21.15.72:0/2777399386 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719000c9d0 con 0x7f71a00afe90 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.051+0000 7f7195ffb640 1 -- 172.21.15.72:0/2777399386 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000cd40 con 0x7f71a00afe90 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.051+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a014fb80 con 0x7f71a00afe90 2023-12-12T09:52:34.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.051+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a014efa0 con 0x7f71a00afe90 2023-12-12T09:52:34.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.052+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a00745d0 con 0x7f71a00afe90 2023-12-12T09:52:34.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.054+0000 7f7195ffb640 1 -- 172.21.15.72:0/2777399386 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7190013420 con 0x7f71a00afe90 2023-12-12T09:52:34.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.055+0000 7f7195ffb640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f718406ae70 0x7f718406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.055+0000 7f7195ffb640 1 -- 172.21.15.72:0/2777399386 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f71900895b0 con 0x7f71a00afe90 2023-12-12T09:52:34.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.055+0000 7f71a546b640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f718406ae70 0x7f718406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.056+0000 7f71a546b640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f718406ae70 0x7f718406d330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f719c035000 tx=0x7f719c03a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:34.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.059+0000 7f7195ffb640 1 -- 172.21.15.72:0/2777399386 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7190054370 con 0x7f71a00afe90 2023-12-12T09:52:34.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.251+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71a00d3820 con 0x7f71a00afe90 2023-12-12T09:52:34.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.252+0000 7f7195ffb640 1 -- 172.21.15.72:0/2777399386 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f7190057990 con 0x7f71a00afe90 2023-12-12T09:52:34.254 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:34.254 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f718406ae70 msgr2=0x7f718406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f718406ae70 0x7f718406d330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f719c035000 tx=0x7f719c03a040 comp rx=0 tx=0).stop 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a00afe90 msgr2=0x7f71a00b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a00afe90 0x7f71a00b0310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7190000ff0 tx=0x7f719000ec80 comp rx=0 tx=0).stop 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 shutdown_connections 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f718406ae70 0x7f718406d330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a00b6d80 0x7f71a00b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a00afe90 0x7f71a00b0310 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 --2- 172.21.15.72:0/2777399386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f718c003970 0x7f71a014c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 >> 172.21.15.72:0/2777399386 conn(0x7f71a009c450 msgr2=0x7f71a006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 shutdown_connections 2023-12-12T09:52:34.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.255+0000 7f71a76cf640 1 -- 172.21.15.72:0/2777399386 wait complete. 2023-12-12T09:52:34.276 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-12T09:52:34.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.468+0000 7f458f35a640 1 Processor -- start 2023-12-12T09:52:34.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.468+0000 7f458f35a640 1 -- start start 2023-12-12T09:52:34.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.469+0000 7f458f35a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f458815d5f0 0x7f45880d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.469+0000 7f458f35a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45880d0200 0x7f45880cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.469+0000 7f458f35a640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45880b2300 con 0x7f45880d3a50 2023-12-12T09:52:34.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.469+0000 7f458f35a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458815e620 con 0x7f45880d0200 2023-12-12T09:52:34.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.469+0000 7f458f35a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458815ea40 con 0x7f458815d5f0 2023-12-12T09:52:34.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.470+0000 7f458c8f5640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f45880d3a50 0x7f45880cfaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42110/0 (socket says 172.21.15.72:42110) 2023-12-12T09:52:34.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.470+0000 7f458c8f5640 1 -- 172.21.15.72:0/3551246614 learned_addr learned my addr 172.21.15.72:0/3551246614 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:34.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.470+0000 7f458d8f7640 1 --2- 172.21.15.72:0/3551246614 >> v2:172.21.15.113:3300/0 conn(0x7f45880d0200 0x7f45880cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.470+0000 7f458d0f6640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f458815d5f0 0x7f45880d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.470+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887082250 0 0) 0x7f45880b2300 con 0x7f45880d3a50 2023-12-12T09:52:34.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.470+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4574003520 con 0x7f45880d3a50 2023-12-12T09:52:34.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2082077126 0 0) 0x7f4574003520 con 0x7f45880d3a50 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f458815d5f0 msgr2=0x7f45880d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f458815d5f0 0x7f45880d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 >> v2:172.21.15.113:3300/0 conn(0x7f45880d0200 msgr2=0x7f45880cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 --2- 172.21.15.72:0/3551246614 >> v2:172.21.15.113:3300/0 conn(0x7f45880d0200 0x7f45880cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45880d3eb0 con 0x7f45880d3a50 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f4578001670 con 0x7f45880d3a50 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 >> v1:172.21.15.72:6789/0 conn(0x7f45880d3a50 legacy=0x7f45880cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f458815d5f0 0x7f4574003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4574003f40 0x7f45740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4574004d00 0x7f4574005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458815e620 con 0x7f4574004d00 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458815ea40 con 0x7f4574003f40 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457400bae0 con 0x7f458815d5f0 2023-12-12T09:52:34.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f458d8f7640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4574004d00 0x7f4574005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f458d0f6640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f458815d5f0 0x7f4574003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f458c8f5640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4574003f40 0x7f45740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f458d8f7640 1 -- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f458815d5f0 msgr2=0x7f4574003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:34.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f458d8f7640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f458815d5f0 0x7f4574003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f458d8f7640 1 -- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4574003f40 msgr2=0x7f45740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:34.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.471+0000 7f458d8f7640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4574003f40 0x7f45740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.472+0000 7f458d8f7640 1 -- 172.21.15.72:0/3551246614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4574003520 con 0x7f4574004d00 2023-12-12T09:52:34.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.472+0000 7f458c8f5640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4574003f40 0x7f45740047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:34.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.472+0000 7f458d8f7640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4574004d00 0x7f4574005180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4580009f90 tx=0x7f45740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bca3b4dc506b69 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:34.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.473+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458002cc20 con 0x7f4574004d00 2023-12-12T09:52:34.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.473+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458000f040 con 0x7f4574004d00 2023-12-12T09:52:34.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.473+0000 7f457ffff640 1 -- 172.21.15.72:0/3551246614 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580034440 con 0x7f4574004d00 2023-12-12T09:52:34.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.473+0000 7f458f35a640 1 -- 172.21.15.72:0/3551246614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4574004d00 msgr2=0x7f4574005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.473+0000 7f458f35a640 1 --2- 172.21.15.72:0/3551246614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4574004d00 0x7f4574005180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4580009f90 tx=0x7f45740057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.473+0000 7f458d8f7640 1 -- 172.21.15.72:0/3551246614 reap_dead start 2023-12-12T09:52:34.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.473+0000 7f458f35a640 1 -- 172.21.15.72:0/3551246614 shutdown_connections 2023-12-12T09:52:34.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.473+0000 7f458f35a640 1 -- 172.21.15.72:0/3551246614 >> 172.21.15.72:0/3551246614 conn(0x7f458809c450 msgr2=0x7f45880d3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:34.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.474+0000 7f458f35a640 1 -- 172.21.15.72:0/3551246614 shutdown_connections 2023-12-12T09:52:34.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.474+0000 7f458f35a640 1 -- 172.21.15.72:0/3551246614 wait complete. 2023-12-12T09:52:34.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.475+0000 7f458f35a640 1 Processor -- start 2023-12-12T09:52:34.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.476+0000 7f458f35a640 1 -- start start 2023-12-12T09:52:34.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458f35a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4574004d00 0x7f45880cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458f35a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4588152410 0x7f45880b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458f35a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458815faf0 0x7f458815ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458f35a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881604b0 con 0x7f4588152410 2023-12-12T09:52:34.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458f35a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45880cb930 con 0x7f458815faf0 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458f35a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45880ca370 con 0x7f4574004d00 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458c8f5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4588152410 0x7f45880b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458c8f5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4588152410 0x7f45880b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53734/0 (socket says 172.21.15.72:53734) 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458c8f5640 1 -- 172.21.15.72:0/1977076996 learned_addr learned my addr 172.21.15.72:0/1977076996 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458d8f7640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458815faf0 0x7f458815ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458d0f6640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4574004d00 0x7f45880cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458c8f5640 1 -- 172.21.15.72:0/1977076996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4574004d00 msgr2=0x7f45880cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458c8f5640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4574004d00 0x7f45880cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458c8f5640 1 -- 172.21.15.72:0/1977076996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458815faf0 msgr2=0x7f458815ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458c8f5640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458815faf0 0x7f458815ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.477+0000 7f458c8f5640 1 -- 172.21.15.72:0/1977076996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45880ca630 con 0x7f4588152410 2023-12-12T09:52:34.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.478+0000 7f458c8f5640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4588152410 0x7f45880b8670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f457800d890 tx=0x7f457800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:34.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.478+0000 7f457dffb640 1 -- 172.21.15.72:0/1977076996 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45780074e0 con 0x7f4588152410 2023-12-12T09:52:34.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.478+0000 7f457dffb640 1 -- 172.21.15.72:0/1977076996 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4578007640 con 0x7f4588152410 2023-12-12T09:52:34.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.478+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45880ca910 con 0x7f4588152410 2023-12-12T09:52:34.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.478+0000 7f457dffb640 1 -- 172.21.15.72:0/1977076996 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4578004a00 con 0x7f4588152410 2023-12-12T09:52:34.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.478+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45880caed0 con 0x7f4588152410 2023-12-12T09:52:34.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.479+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45880745d0 con 0x7f4588152410 2023-12-12T09:52:34.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.480+0000 7f457dffb640 1 -- 172.21.15.72:0/1977076996 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4578004b60 con 0x7f4588152410 2023-12-12T09:52:34.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.482+0000 7f457dffb640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f456c06ae70 0x7f456c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:34.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.482+0000 7f457dffb640 1 -- 172.21.15.72:0/1977076996 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4578089710 con 0x7f4588152410 2023-12-12T09:52:34.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.482+0000 7f458d0f6640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f456c06ae70 0x7f456c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:34.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.483+0000 7f458d0f6640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f456c06ae70 0x7f456c06d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4584006fd0 tx=0x7f4584008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:34.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.486+0000 7f457dffb640 1 -- 172.21.15.72:0/1977076996 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4578054430 con 0x7f4588152410 2023-12-12T09:52:34.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:34.714+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f45880d06d0 con 0x7f4588152410 2023-12-12T09:52:35.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.420+0000 7f457dffb640 1 -- 172.21.15.72:0/1977076996 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4578057a50 con 0x7f4588152410 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.423+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f456c06ae70 msgr2=0x7f456c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.423+0000 7f458f35a640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f456c06ae70 0x7f456c06d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4584006fd0 tx=0x7f4584008040 comp rx=0 tx=0).stop 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4588152410 msgr2=0x7f45880b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4588152410 0x7f45880b8670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f457800d890 tx=0x7f457800dd50 comp rx=0 tx=0).stop 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 shutdown_connections 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f456c06ae70 0x7f456c06d330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f458815faf0 0x7f458815ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4588152410 0x7f45880b8670 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 --2- 172.21.15.72:0/1977076996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4574004d00 0x7f45880cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 >> 172.21.15.72:0/1977076996 conn(0x7f458809c450 msgr2=0x7f458806b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:35.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 shutdown_connections 2023-12-12T09:52:35.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.424+0000 7f458f35a640 1 -- 172.21.15.72:0/1977076996 wait complete. 2023-12-12T09:52:35.446 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-12-12T09:52:35.447 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-12T09:52:35.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.678+0000 7fbda0a89640 1 Processor -- start 2023-12-12T09:52:35.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.678+0000 7fbda0a89640 1 -- start start 2023-12-12T09:52:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbda0a89640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbd9c143780 0x7fbd9c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbda0a89640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0ce720 0x7fbd9c0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbda0a89640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c072510 con 0x7fbd9c0720c0 2023-12-12T09:52:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbda0a89640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c0b4d80 con 0x7fbd9c143780 2023-12-12T09:52:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbda0a89640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0b1d90 con 0x7fbd9c0ce720 2023-12-12T09:52:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd9a59c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbd9c143780 0x7fbd9c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd9a59c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbd9c143780 0x7fbd9c071b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40152/0 (socket says 172.21.15.72:40152) 2023-12-12T09:52:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd9a59c640 1 -- 172.21.15.72:0/2484154179 learned_addr learned my addr 172.21.15.72:0/2484154179 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd93fff640 1 --1- 172.21.15.72:0/2484154179 >> v1:172.21.15.72:6789/0 conn(0x7fbd9c0720c0 0x7fbd9c0ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52324/0 (socket says 172.21.15.72:52324) 2023-12-12T09:52:35.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd9a59c640 1 -- 172.21.15.72:0/2484154179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0ce720 msgr2=0x7fbd9c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:35.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd9ad9d640 1 --2- 172.21.15.72:0/2484154179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0ce720 0x7fbd9c0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:35.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd9a59c640 1 --2- 172.21.15.72:0/2484154179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0ce720 0x7fbd9c0b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd99d9b640 1 -- 172.21.15.72:0/2484154179 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190614132 0 0) 0x7fbd84001110 con 0x7fbd9c0720c0 2023-12-12T09:52:35.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd9a59c640 1 -- 172.21.15.72:0/2484154179 >> v1:172.21.15.72:6789/0 conn(0x7fbd9c0720c0 legacy=0x7fbd9c0ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:35.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.680+0000 7fbd9a59c640 1 -- 172.21.15.72:0/2484154179 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0b2910 con 0x7fbd9c143780 2023-12-12T09:52:35.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.681+0000 7fbd9a59c640 1 --2- 172.21.15.72:0/2484154179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c071b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009870 tx=0x7fbd8c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef13ac99ad06383d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbd99d9b640 1 -- 172.21.15.72:0/2484154179 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c030840 con 0x7fbd9c143780 2023-12-12T09:52:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbd99d9b640 1 -- 172.21.15.72:0/2484154179 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c0309a0 con 0x7fbd9c143780 2023-12-12T09:52:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbd99d9b640 1 -- 172.21.15.72:0/2484154179 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c030d30 con 0x7fbd9c143780 2023-12-12T09:52:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbda0a89640 1 -- 172.21.15.72:0/2484154179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 msgr2=0x7fbd9c071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbda0a89640 1 --2- 172.21.15.72:0/2484154179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c071b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009870 tx=0x7fbd8c02b980 comp rx=0 tx=0).stop 2023-12-12T09:52:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbda0a89640 1 -- 172.21.15.72:0/2484154179 shutdown_connections 2023-12-12T09:52:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbda0a89640 1 --2- 172.21.15.72:0/2484154179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0ce720 0x7fbd9c0b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbda0a89640 1 --2- 172.21.15.72:0/2484154179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c071b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbda0a89640 1 -- 172.21.15.72:0/2484154179 >> 172.21.15.72:0/2484154179 conn(0x7fbd9c06c430 msgr2=0x7fbd9c06c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbda0a89640 1 -- 172.21.15.72:0/2484154179 shutdown_connections 2023-12-12T09:52:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.682+0000 7fbda0a89640 1 -- 172.21.15.72:0/2484154179 wait complete. 2023-12-12T09:52:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.684+0000 7fbda0a89640 1 Processor -- start 2023-12-12T09:52:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.684+0000 7fbda0a89640 1 -- start start 2023-12-12T09:52:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbda0a89640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0720c0 0x7fbd9c1f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbda0a89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd9c0ce720 0x7fbd9c14cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbda0a89640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c13a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbda0a89640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c14db70 con 0x7fbd9c143780 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbda0a89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c14d5b0 con 0x7fbd9c0ce720 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbda0a89640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c14d890 con 0x7fbd9c0720c0 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbd9ad9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c13a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbd9ad9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c13a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40158/0 (socket says 172.21.15.72:40158) 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.685+0000 7fbd9ad9d640 1 -- 172.21.15.72:0/1562185955 learned_addr learned my addr 172.21.15.72:0/1562185955 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd93fff640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd9c0ce720 0x7fbd9c14cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd9a59c640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0720c0 0x7fbd9c1f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd9ad9d640 1 -- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0720c0 msgr2=0x7fbd9c1f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd9ad9d640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0720c0 0x7fbd9c1f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd9ad9d640 1 -- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd9c0ce720 msgr2=0x7fbd9c14cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd9ad9d640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd9c0ce720 0x7fbd9c14cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd9ad9d640 1 -- 172.21.15.72:0/1562185955 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c0094d0 con 0x7fbd9c143780 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd93fff640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd9c0ce720 0x7fbd9c14cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd9a59c640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0720c0 0x7fbd9c1f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd9ad9d640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c13a750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400cd90 tx=0x7fbd9400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd92ffd640 1 -- 172.21.15.72:0/1562185955 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94003b90 con 0x7fbd9c143780 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd92ffd640 1 -- 172.21.15.72:0/1562185955 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd94003d10 con 0x7fbd9c143780 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbd92ffd640 1 -- 172.21.15.72:0/1562185955 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94004260 con 0x7fbd9c143780 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.686+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c13c410 con 0x7fbd9c143780 2023-12-12T09:52:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.687+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c14d1a0 con 0x7fbd9c143780 2023-12-12T09:52:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.688+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c13c930 con 0x7fbd9c143780 2023-12-12T09:52:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.689+0000 7fbd92ffd640 1 -- 172.21.15.72:0/1562185955 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd940043c0 con 0x7fbd9c143780 2023-12-12T09:52:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.689+0000 7fbd92ffd640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbd7c06ae40 0x7fbd7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.689+0000 7fbd92ffd640 1 -- 172.21.15.72:0/1562185955 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd94012070 con 0x7fbd9c143780 2023-12-12T09:52:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.690+0000 7fbd9a59c640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbd7c06ae40 0x7fbd7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.690+0000 7fbd9a59c640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbd7c06ae40 0x7fbd7c06d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c036000 tx=0x7fbd8c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.693+0000 7fbd92ffd640 1 -- 172.21.15.72:0/1562185955 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd9c13c930 con 0x7fbd9c143780 2023-12-12T09:52:35.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:35.922+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbd9c13c930 con 0x7fbd9c143780 2023-12-12T09:52:36.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.423+0000 7fbd92ffd640 1 -- 172.21.15.72:0/1562185955 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbd9c13c930 con 0x7fbd9c143780 2023-12-12T09:52:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.427+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbd7c06ae40 msgr2=0x7fbd7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.427+0000 7fbda0a89640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbd7c06ae40 0x7fbd7c06d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c036000 tx=0x7fbd8c03b040 comp rx=0 tx=0).stop 2023-12-12T09:52:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.427+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 msgr2=0x7fbd9c13a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.427+0000 7fbda0a89640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c13a750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400cd90 tx=0x7fbd9400a660 comp rx=0 tx=0).stop 2023-12-12T09:52:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.427+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 shutdown_connections 2023-12-12T09:52:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.427+0000 7fbda0a89640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbd7c06ae40 0x7fbd7c06d300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.428+0000 7fbda0a89640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd9c143780 0x7fbd9c13a750 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.428+0000 7fbda0a89640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd9c0ce720 0x7fbd9c14cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.428+0000 7fbda0a89640 1 --2- 172.21.15.72:0/1562185955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd9c0720c0 0x7fbd9c1f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.428+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 >> 172.21.15.72:0/1562185955 conn(0x7fbd9c06c430 msgr2=0x7fbd9c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.428+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 shutdown_connections 2023-12-12T09:52:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.428+0000 7fbda0a89640 1 -- 172.21.15.72:0/1562185955 wait complete. 2023-12-12T09:52:36.450 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-12T09:52:36.451 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:52:36.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.652+0000 7f162404b640 1 Processor -- start 2023-12-12T09:52:36.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.652+0000 7f162404b640 1 -- start start 2023-12-12T09:52:36.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.654+0000 7f162404b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f161c06d520 0x7f161c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:36.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.654+0000 7f162404b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161c0cde40 con 0x7f161c15f260 2023-12-12T09:52:36.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.654+0000 7f162404b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161c0cd230 con 0x7f161c0708d0 2023-12-12T09:52:36.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.654+0000 7f162404b640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c0cd040 con 0x7f161c06d520 2023-12-12T09:52:36.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.654+0000 7f1621de7640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f161c15f260 0x7f161c0701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52326/0 (socket says 172.21.15.72:52326) 2023-12-12T09:52:36.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.654+0000 7f1621de7640 1 -- 172.21.15.72:0/954621408 learned_addr learned my addr 172.21.15.72:0/954621408 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:36.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.654+0000 7f16225e8640 1 --2- 172.21.15.72:0/954621408 >> v2:172.21.15.72:3300/0 conn(0x7f161c06d520 0x7f161c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:36.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.655+0000 7f16225e8640 1 -- 172.21.15.72:0/954621408 >> v1:172.21.15.113:6789/0 conn(0x7f161c0708d0 legacy=0x7f161c1691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:36.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.655+0000 7f1620de5640 1 -- 172.21.15.72:0/954621408 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763344911 0 0) 0x7f161c0cde40 con 0x7f161c15f260 2023-12-12T09:52:36.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.655+0000 7f16225e8640 1 -- 172.21.15.72:0/954621408 >> v1:172.21.15.72:6789/0 conn(0x7f161c15f260 legacy=0x7f161c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:36.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.655+0000 7f16225e8640 1 -- 172.21.15.72:0/954621408 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c0cd510 con 0x7f161c06d520 2023-12-12T09:52:36.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.656+0000 7f16225e8640 1 --2- 172.21.15.72:0/954621408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c06d520 0x7f161c0ccb00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16140097a0 tx=0x7f161402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc101832990c693 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:36.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f1620de5640 1 -- 172.21.15.72:0/954621408 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614031830 con 0x7f161c06d520 2023-12-12T09:52:36.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f1620de5640 1 -- 172.21.15.72:0/954621408 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614031990 con 0x7f161c06d520 2023-12-12T09:52:36.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f1620de5640 1 -- 172.21.15.72:0/954621408 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614031d20 con 0x7f161c06d520 2023-12-12T09:52:36.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f162404b640 1 -- 172.21.15.72:0/954621408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c06d520 msgr2=0x7f161c0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:36.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f162404b640 1 --2- 172.21.15.72:0/954621408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c06d520 0x7f161c0ccb00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16140097a0 tx=0x7f161402b8d0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f162404b640 1 -- 172.21.15.72:0/954621408 shutdown_connections 2023-12-12T09:52:36.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f162404b640 1 --2- 172.21.15.72:0/954621408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c06d520 0x7f161c0ccb00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f162404b640 1 -- 172.21.15.72:0/954621408 >> 172.21.15.72:0/954621408 conn(0x7f161c0b0a90 msgr2=0x7f161c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:36.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f162404b640 1 -- 172.21.15.72:0/954621408 shutdown_connections 2023-12-12T09:52:36.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.657+0000 7f162404b640 1 -- 172.21.15.72:0/954621408 wait complete. 2023-12-12T09:52:36.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.659+0000 7f162404b640 1 Processor -- start 2023-12-12T09:52:36.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.659+0000 7f162404b640 1 -- start start 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.660+0000 7f162404b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161c06d520 0x7f161c151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.660+0000 7f162404b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f161c0708d0 0x7f161c0b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.660+0000 7f162404b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c15f260 0x7f161c160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.660+0000 7f162404b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c151ec0 con 0x7f161c15f260 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.660+0000 7f162404b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0b2440 con 0x7f161c0708d0 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.660+0000 7f162404b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0b2720 con 0x7f161c06d520 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.660+0000 7f16225e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c15f260 0x7f161c160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16225e8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c15f260 0x7f161c160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40184/0 (socket says 172.21.15.72:40184) 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16225e8640 1 -- 172.21.15.72:0/2169093881 learned_addr learned my addr 172.21.15.72:0/2169093881 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f1621de7640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161c06d520 0x7f161c151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:36.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16215e6640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f161c0708d0 0x7f161c0b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:36.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16225e8640 1 -- 172.21.15.72:0/2169093881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161c06d520 msgr2=0x7f161c151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:36.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16225e8640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161c06d520 0x7f161c151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16225e8640 1 -- 172.21.15.72:0/2169093881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f161c0708d0 msgr2=0x7f161c0b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:36.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16225e8640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f161c0708d0 0x7f161c0b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16225e8640 1 -- 172.21.15.72:0/2169093881 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614009400 con 0x7f161c15f260 2023-12-12T09:52:36.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.661+0000 7f16225e8640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c15f260 0x7f161c160810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16140098c0 tx=0x7f16140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:36.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.662+0000 7f161a7fc640 1 -- 172.21.15.72:0/2169093881 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614004230 con 0x7f161c15f260 2023-12-12T09:52:36.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.662+0000 7f161a7fc640 1 -- 172.21.15.72:0/2169093881 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614004390 con 0x7f161c15f260 2023-12-12T09:52:36.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.662+0000 7f161a7fc640 1 -- 172.21.15.72:0/2169093881 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161403a920 con 0x7f161c15f260 2023-12-12T09:52:36.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.662+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c0b2a00 con 0x7f161c15f260 2023-12-12T09:52:36.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.662+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c0a87d0 con 0x7f161c15f260 2023-12-12T09:52:36.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.663+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c0745d0 con 0x7f161c15f260 2023-12-12T09:52:36.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.664+0000 7f161a7fc640 1 -- 172.21.15.72:0/2169093881 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f161403aa80 con 0x7f161c15f260 2023-12-12T09:52:36.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.664+0000 7f161a7fc640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f160006ae40 0x7f160006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:36.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.665+0000 7f161a7fc640 1 -- 172.21.15.72:0/2169093881 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f16140ac030 con 0x7f161c15f260 2023-12-12T09:52:36.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.665+0000 7f1621de7640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f160006ae40 0x7f160006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:36.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.666+0000 7f1621de7640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f160006ae40 0x7f160006d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1610007ed0 tx=0x7f1610007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:36.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.669+0000 7f161a7fc640 1 -- 172.21.15.72:0/2169093881 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1614076e80 con 0x7f161c15f260 2023-12-12T09:52:36.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.904+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f161c0b4d40 con 0x7f161c15f260 2023-12-12T09:52:36.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.905+0000 7f161a7fc640 1 -- 172.21.15.72:0/2169093881 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f161403ad30 con 0x7f161c15f260 2023-12-12T09:52:36.907 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:36.907 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:35.924557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:52:36.907 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.907+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f160006ae40 msgr2=0x7f160006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.907+0000 7f162404b640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f160006ae40 0x7f160006d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1610007ed0 tx=0x7f1610007e60 comp rx=0 tx=0).stop 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.907+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c15f260 msgr2=0x7f161c160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c15f260 0x7f161c160810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16140098c0 tx=0x7f16140046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 shutdown_connections 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f160006ae40 0x7f160006d300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f161c15f260 0x7f161c160810 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f161c0708d0 0x7f161c0b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 --2- 172.21.15.72:0/2169093881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161c06d520 0x7f161c151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 >> 172.21.15.72:0/2169093881 conn(0x7f161c0b0a90 msgr2=0x7f161c0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 shutdown_connections 2023-12-12T09:52:36.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:36.908+0000 7f162404b640 1 -- 172.21.15.72:0/2169093881 wait complete. 2023-12-12T09:52:36.929 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:37.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.134+0000 7fc247890640 1 Processor -- start 2023-12-12T09:52:37.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.134+0000 7fc247890640 1 -- start start 2023-12-12T09:52:37.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.135+0000 7fc247890640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc240143e00 0x7fc2400b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.135+0000 7fc247890640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc240072180 0x7fc2400b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.135+0000 7fc247890640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 0x7fc2400b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.135+0000 7fc247890640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2400b3bb0 con 0x7fc240072180 2023-12-12T09:52:37.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.135+0000 7fc247890640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2401441e0 con 0x7fc240143e00 2023-12-12T09:52:37.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.135+0000 7fc247890640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc240072680 con 0x7fc24006c410 2023-12-12T09:52:37.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc244e2b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc240072180 0x7fc2400b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc244e2b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc240072180 0x7fc2400b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40198/0 (socket says 172.21.15.72:40198) 2023-12-12T09:52:37.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc244e2b640 1 -- 172.21.15.72:0/131206209 learned_addr learned my addr 172.21.15.72:0/131206209 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:37.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc245e2d640 1 --2- 172.21.15.72:0/131206209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 0x7fc2400b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc24562c640 1 --2- 172.21.15.72:0/131206209 >> v2:172.21.15.113:3300/0 conn(0x7fc240143e00 0x7fc2400b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc244e2b640 1 -- 172.21.15.72:0/131206209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 msgr2=0x7fc2400b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:37.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc244e2b640 1 --2- 172.21.15.72:0/131206209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 0x7fc2400b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc244e2b640 1 -- 172.21.15.72:0/131206209 >> v2:172.21.15.113:3300/0 conn(0x7fc240143e00 msgr2=0x7fc2400b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc244e2b640 1 --2- 172.21.15.72:0/131206209 >> v2:172.21.15.113:3300/0 conn(0x7fc240143e00 0x7fc2400b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.136+0000 7fc244e2b640 1 -- 172.21.15.72:0/131206209 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2400b1e30 con 0x7fc240072180 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.137+0000 7fc244e2b640 1 --2- 172.21.15.72:0/131206209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240072180 0x7fc2400b4580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc24006b760 tx=0x7fc23002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171aa367cf51cd6a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc237fff640 1 -- 172.21.15.72:0/131206209 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230031870 con 0x7fc240072180 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc237fff640 1 -- 172.21.15.72:0/131206209 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2300319d0 con 0x7fc240072180 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc237fff640 1 -- 172.21.15.72:0/131206209 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230031d60 con 0x7fc240072180 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 -- 172.21.15.72:0/131206209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240072180 msgr2=0x7fc2400b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 --2- 172.21.15.72:0/131206209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240072180 0x7fc2400b4580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc24006b760 tx=0x7fc23002b910 comp rx=0 tx=0).stop 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 -- 172.21.15.72:0/131206209 shutdown_connections 2023-12-12T09:52:37.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 --2- 172.21.15.72:0/131206209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 0x7fc2400b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 --2- 172.21.15.72:0/131206209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240072180 0x7fc2400b4580 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 --2- 172.21.15.72:0/131206209 >> v2:172.21.15.113:3300/0 conn(0x7fc240143e00 0x7fc2400b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 -- 172.21.15.72:0/131206209 >> 172.21.15.72:0/131206209 conn(0x7fc2400b0a90 msgr2=0x7fc2400b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:37.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 -- 172.21.15.72:0/131206209 shutdown_connections 2023-12-12T09:52:37.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.138+0000 7fc247890640 1 -- 172.21.15.72:0/131206209 wait complete. 2023-12-12T09:52:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.140+0000 7fc247890640 1 Processor -- start 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.140+0000 7fc247890640 1 -- start start 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.141+0000 7fc247890640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 0x7fc24013c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.141+0000 7fc247890640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc240072180 0x7fc24013b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.141+0000 7fc247890640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240143e00 0x7fc24013b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.141+0000 7fc247890640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400cd530 con 0x7fc240143e00 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.141+0000 7fc247890640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400abf30 con 0x7fc240072180 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.141+0000 7fc247890640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400cc8f0 con 0x7fc24006c410 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.141+0000 7fc245e2d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240143e00 0x7fc24013b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc245e2d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240143e00 0x7fc24013b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40200/0 (socket says 172.21.15.72:40200) 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc245e2d640 1 -- 172.21.15.72:0/4084410790 learned_addr learned my addr 172.21.15.72:0/4084410790 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc24562c640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 0x7fc24013c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc244e2b640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc240072180 0x7fc24013b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc245e2d640 1 -- 172.21.15.72:0/4084410790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 msgr2=0x7fc24013c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc245e2d640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 0x7fc24013c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc245e2d640 1 -- 172.21.15.72:0/4084410790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc240072180 msgr2=0x7fc24013b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc245e2d640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc240072180 0x7fc24013b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc245e2d640 1 -- 172.21.15.72:0/4084410790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2300094d0 con 0x7fc240143e00 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc245e2d640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240143e00 0x7fc24013b9e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc23800d5a0 tx=0x7fc23800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.142+0000 7fc235ffb640 1 -- 172.21.15.72:0/4084410790 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238014620 con 0x7fc240143e00 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.143+0000 7fc235ffb640 1 -- 172.21.15.72:0/4084410790 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc238014780 con 0x7fc240143e00 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.143+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2400cd950 con 0x7fc240143e00 2023-12-12T09:52:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.143+0000 7fc235ffb640 1 -- 172.21.15.72:0/4084410790 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238014b30 con 0x7fc240143e00 2023-12-12T09:52:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.143+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2400cce70 con 0x7fc240143e00 2023-12-12T09:52:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.144+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2400cd670 con 0x7fc240143e00 2023-12-12T09:52:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.145+0000 7fc235ffb640 1 -- 172.21.15.72:0/4084410790 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc238014c90 con 0x7fc240143e00 2023-12-12T09:52:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.146+0000 7fc235ffb640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc22406ae40 0x7fc22406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.146+0000 7fc235ffb640 1 -- 172.21.15.72:0/4084410790 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc238089990 con 0x7fc240143e00 2023-12-12T09:52:37.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.146+0000 7fc24562c640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc22406ae40 0x7fc22406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.146+0000 7fc24562c640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc22406ae40 0x7fc22406d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009a60 tx=0x7fc23c003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.150+0000 7fc235ffb640 1 -- 172.21.15.72:0/4084410790 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2400cd670 con 0x7fc240143e00 2023-12-12T09:52:37.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.340+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2400cd670 con 0x7fc240143e00 2023-12-12T09:52:37.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.344+0000 7fc235ffb640 1 -- 172.21.15.72:0/4084410790 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fc2400cd670 con 0x7fc240143e00 2023-12-12T09:52:37.345 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:37.346 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:37.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc22406ae40 msgr2=0x7fc22406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc22406ae40 0x7fc22406d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009a60 tx=0x7fc23c003990 comp rx=0 tx=0).stop 2023-12-12T09:52:37.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240143e00 msgr2=0x7fc24013b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240143e00 0x7fc24013b9e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc23800d5a0 tx=0x7fc23800da60 comp rx=0 tx=0).stop 2023-12-12T09:52:37.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 shutdown_connections 2023-12-12T09:52:37.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc22406ae40 0x7fc22406d300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc240143e00 0x7fc24013b9e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc240072180 0x7fc24013b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 --2- 172.21.15.72:0/4084410790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24006c410 0x7fc24013c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 >> 172.21.15.72:0/4084410790 conn(0x7fc2400b0a90 msgr2=0x7fc240070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:37.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 shutdown_connections 2023-12-12T09:52:37.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.346+0000 7fc247890640 1 -- 172.21.15.72:0/4084410790 wait complete. 2023-12-12T09:52:37.368 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-12T09:52:37.368 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:52:37.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.558+0000 7f699481c640 1 Processor -- start 2023-12-12T09:52:37.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.558+0000 7f699481c640 1 -- start start 2023-12-12T09:52:37.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.559+0000 7f699481c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f699006c410 0x7f699015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.559+0000 7f699481c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 0x7f69900cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.559+0000 7f699481c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699006ca40 con 0x7f6990157a60 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.559+0000 7f699481c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699015e550 con 0x7f699006c410 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.559+0000 7f699481c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6990071cd0 con 0x7f699015e0e0 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698e59c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6990157a60 0x7f699006de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52336/0 (socket says 172.21.15.72:52336) 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698e59c640 1 -- 172.21.15.72:0/140362547 learned_addr learned my addr 172.21.15.72:0/140362547 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698dd9b640 1 --2- 172.21.15.72:0/140362547 >> v2:172.21.15.113:3300/0 conn(0x7f699006c410 0x7f699015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698ed9d640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 0x7f69900cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583327244 0 0) 0x7f699006ca40 con 0x7f6990157a60 2023-12-12T09:52:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6978003520 con 0x7f6990157a60 2023-12-12T09:52:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636486194 0 0) 0x7f6978003520 con 0x7f6990157a60 2023-12-12T09:52:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 msgr2=0x7f69900cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 0x7f69900cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 >> v2:172.21.15.113:3300/0 conn(0x7f699006c410 msgr2=0x7f699015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 --2- 172.21.15.72:0/140362547 >> v2:172.21.15.113:3300/0 conn(0x7f699006c410 0x7f699015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6990070f10 con 0x7f6990157a60 2023-12-12T09:52:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.560+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f6980001670 con 0x7f6990157a60 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698ed9d640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 0x7f69900cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698dd9b640 1 --2- 172.21.15.72:0/140362547 >> v2:172.21.15.113:3300/0 conn(0x7f699006c410 0x7f699015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 >> v1:172.21.15.72:6789/0 conn(0x7f6990157a60 legacy=0x7f699006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698d59a640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 0x7f6978003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698d59a640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978003f60 0x7f69780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698d59a640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6978004d00 0x7f6978005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699015e550 con 0x7f6978004d00 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6990071cd0 con 0x7f6978003f60 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697800bae0 con 0x7f699015e0e0 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698ed9d640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6978004d00 0x7f6978005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698e59c640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 0x7f6978003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698dd9b640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978003f60 0x7f69780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698ed9d640 1 -- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 msgr2=0x7f6978003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698ed9d640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 0x7f6978003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698ed9d640 1 -- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978003f60 msgr2=0x7f69780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698ed9d640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978003f60 0x7f69780047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698ed9d640 1 -- 172.21.15.72:0/140362547 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6978003520 con 0x7f6978004d00 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.561+0000 7f698e59c640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f699015e0e0 0x7f6978003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.562+0000 7f698ed9d640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6978004d00 0x7f6978005180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69880073e0 tx=0x7f69780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8d144ade2ba229 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698802cd30 con 0x7f6978004d00 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698802ce90 con 0x7f6978004d00 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f698d59a640 1 -- 172.21.15.72:0/140362547 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988034640 con 0x7f6978004d00 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f699481c640 1 -- 172.21.15.72:0/140362547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6978004d00 msgr2=0x7f6978005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f699481c640 1 --2- 172.21.15.72:0/140362547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6978004d00 0x7f6978005180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69880073e0 tx=0x7f69780057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f698ed9d640 1 -- 172.21.15.72:0/140362547 reap_dead start 2023-12-12T09:52:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f699481c640 1 -- 172.21.15.72:0/140362547 shutdown_connections 2023-12-12T09:52:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f699481c640 1 -- 172.21.15.72:0/140362547 >> 172.21.15.72:0/140362547 conn(0x7f69900b0a90 msgr2=0x7f69900b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f699481c640 1 -- 172.21.15.72:0/140362547 shutdown_connections 2023-12-12T09:52:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.563+0000 7f699481c640 1 -- 172.21.15.72:0/140362547 wait complete. 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.565+0000 7f699481c640 1 Processor -- start 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.565+0000 7f699481c640 1 -- start start 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f699481c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978004d00 0x7f699014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f699481c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6990158c50 0x7f69900b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f699481c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69900afa50 0x7f69900afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f699481c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69900b0410 con 0x7f69900afa50 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f699481c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699015a0b0 con 0x7f6978004d00 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f699481c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69900b4880 con 0x7f6990158c50 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f698ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69900afa50 0x7f69900afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f698ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69900afa50 0x7f69900afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40218/0 (socket says 172.21.15.72:40218) 2023-12-12T09:52:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.566+0000 7f698ed9d640 1 -- 172.21.15.72:0/83407956 learned_addr learned my addr 172.21.15.72:0/83407956 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698e59c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978004d00 0x7f699014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698dd9b640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6990158c50 0x7f69900b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698ed9d640 1 -- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6990158c50 msgr2=0x7f69900b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698ed9d640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6990158c50 0x7f69900b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698ed9d640 1 -- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978004d00 msgr2=0x7f699014e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698ed9d640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978004d00 0x7f699014e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698ed9d640 1 -- 172.21.15.72:0/83407956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69901592f0 con 0x7f69900afa50 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698e59c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978004d00 0x7f699014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f698ed9d640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69900afa50 0x7f69900afed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69880073e0 tx=0x7f6988004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f697effd640 1 -- 172.21.15.72:0/83407956 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988046070 con 0x7f69900afa50 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f697effd640 1 -- 172.21.15.72:0/83407956 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698802cb30 con 0x7f69900afa50 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f697effd640 1 -- 172.21.15.72:0/83407956 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988003b90 con 0x7f69900afa50 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69901595d0 con 0x7f69900afa50 2023-12-12T09:52:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.567+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6990159b90 con 0x7f69900afa50 2023-12-12T09:52:37.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.568+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69900745d0 con 0x7f69900afa50 2023-12-12T09:52:37.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.569+0000 7f697effd640 1 -- 172.21.15.72:0/83407956 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f698803b050 con 0x7f69900afa50 2023-12-12T09:52:37.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.570+0000 7f697effd640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f697006ae70 0x7f697006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:37.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.570+0000 7f697effd640 1 -- 172.21.15.72:0/83407956 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69880ab1a0 con 0x7f69900afa50 2023-12-12T09:52:37.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.570+0000 7f698e59c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f697006ae70 0x7f697006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:37.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.571+0000 7f698e59c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f697006ae70 0x7f697006d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6980001eb0 tx=0x7f6980009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:37.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.574+0000 7f697effd640 1 -- 172.21.15.72:0/83407956 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6988075f60 con 0x7f69900afa50 2023-12-12T09:52:37.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.746+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69900717b0 con 0x7f697006ae70 2023-12-12T09:52:37.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.780+0000 7f697effd640 1 -- 172.21.15.72:0/83407956 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345154 (secure 0 0 0) 0x7f698000d040 con 0x7f697006ae70 2023-12-12T09:52:37.789 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:37.792 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:52:37.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f697006ae70 msgr2=0x7f697006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f697006ae70 0x7f697006d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6980001eb0 tx=0x7f6980009040 comp rx=0 tx=0).stop 2023-12-12T09:52:37.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69900afa50 msgr2=0x7f69900afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69900afa50 0x7f69900afed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69880073e0 tx=0x7f6988004ab0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 shutdown_connections 2023-12-12T09:52:37.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f697006ae70 0x7f697006d330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f69900afa50 0x7f69900afed0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6990158c50 0x7f69900b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 --2- 172.21.15.72:0/83407956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6978004d00 0x7f699014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 >> 172.21.15.72:0/83407956 conn(0x7f69900b0a90 msgr2=0x7f69900700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:37.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 shutdown_connections 2023-12-12T09:52:37.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:37.786+0000 7f699481c640 1 -- 172.21.15.72:0/83407956 wait complete. 2023-12-12T09:52:37.807 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-12-12T09:52:37.554883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3196,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3272704,"data_stored":1985289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004648"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070257+0000","last_change":"2023-12-12T09:52:07.005949+0000","last_active":"2023-12-12T09:52:12.070257+0000","last_peered":"2023-12-12T09:52:12.070257+0000","last_clean":"2023-12-12T09:52:12.070257+0000","last_became_active":"2023-12-12T09:52:07.005677+0000","last_became_peered":"2023-12-12T09:52:07.005677+0000","last_unstale":"2023-12-12T09:52:12.070257+0000","last_undegraded":"2023-12-12T09:52:12.070257+0000","last_fullsized":"2023-12-12T09:52:12.070257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:31.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.058906+0000","last_change":"2023-12-12T09:52:08.029768+0000","last_active":"2023-12-12T09:52:12.058906+0000","last_peered":"2023-12-12T09:52:12.058906+0000","last_clean":"2023-12-12T09:52:12.058906+0000","last_became_active":"2023-12-12T09:52:08.029493+0000","last_became_peered":"2023-12-12T09:52:08.029493+0000","last_unstale":"2023-12-12T09:52:12.058906+0000","last_undegraded":"2023-12-12T09:52:12.058906+0000","last_fullsized":"2023-12-12T09:52:12.058906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283436+0000","last_change":"2023-12-12T09:52:12.106967+0000","last_active":"2023-12-12T09:52:13.283436+0000","last_peered":"2023-12-12T09:52:13.283436+0000","last_clean":"2023-12-12T09:52:13.283436+0000","last_became_active":"2023-12-12T09:52:12.106583+0000","last_became_peered":"2023-12-12T09:52:12.106583+0000","last_unstale":"2023-12-12T09:52:13.283436+0000","last_undegraded":"2023-12-12T09:52:13.283436+0000","last_fullsized":"2023-12-12T09:52:13.283436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:02.585452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059029+0000","last_change":"2023-12-12T09:52:06.990826+0000","last_active":"2023-12-12T09:52:12.059029+0000","last_peered":"2023-12-12T09:52:12.059029+0000","last_clean":"2023-12-12T09:52:12.059029+0000","last_became_active":"2023-12-12T09:52:06.990515+0000","last_became_peered":"2023-12-12T09:52:06.990515+0000","last_unstale":"2023-12-12T09:52:12.059029+0000","last_undegraded":"2023-12-12T09:52:12.059029+0000","last_fullsized":"2023-12-12T09:52:12.059029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:19.282948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070081+0000","last_change":"2023-12-12T09:52:08.023719+0000","last_active":"2023-12-12T09:52:12.070081+0000","last_peered":"2023-12-12T09:52:12.070081+0000","last_clean":"2023-12-12T09:52:12.070081+0000","last_became_active":"2023-12-12T09:52:08.023372+0000","last_became_peered":"2023-12-12T09:52:08.023372+0000","last_unstale":"2023-12-12T09:52:12.070081+0000","last_undegraded":"2023-12-12T09:52:12.070081+0000","last_fullsized":"2023-12-12T09:52:12.070081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281424+0000","last_change":"2023-12-12T09:52:12.082594+0000","last_active":"2023-12-12T09:52:13.281424+0000","last_peered":"2023-12-12T09:52:13.281424+0000","last_clean":"2023-12-12T09:52:13.281424+0000","last_became_active":"2023-12-12T09:52:12.081951+0000","last_became_peered":"2023-12-12T09:52:12.081951+0000","last_unstale":"2023-12-12T09:52:13.281424+0000","last_undegraded":"2023-12-12T09:52:13.281424+0000","last_fullsized":"2023-12-12T09:52:13.281424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:45.079238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104327+0000","last_change":"2023-12-12T09:52:07.014029+0000","last_active":"2023-12-12T09:52:12.104327+0000","last_peered":"2023-12-12T09:52:12.104327+0000","last_clean":"2023-12-12T09:52:12.104327+0000","last_became_active":"2023-12-12T09:52:07.013726+0000","last_became_peered":"2023-12-12T09:52:07.013726+0000","last_unstale":"2023-12-12T09:52:12.104327+0000","last_undegraded":"2023-12-12T09:52:12.104327+0000","last_fullsized":"2023-12-12T09:52:12.104327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:46.546348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074859+0000","last_change":"2023-12-12T09:52:08.024974+0000","last_active":"2023-12-12T09:52:12.074859+0000","last_peered":"2023-12-12T09:52:12.074859+0000","last_clean":"2023-12-12T09:52:12.074859+0000","last_became_active":"2023-12-12T09:52:08.024739+0000","last_became_peered":"2023-12-12T09:52:08.024739+0000","last_unstale":"2023-12-12T09:52:12.074859+0000","last_undegraded":"2023-12-12T09:52:12.074859+0000","last_fullsized":"2023-12-12T09:52:12.074859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.950798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281819+0000","last_change":"2023-12-12T09:52:12.082902+0000","last_active":"2023-12-12T09:52:13.281819+0000","last_peered":"2023-12-12T09:52:13.281819+0000","last_clean":"2023-12-12T09:52:13.281819+0000","last_became_active":"2023-12-12T09:52:12.082411+0000","last_became_peered":"2023-12-12T09:52:12.082411+0000","last_unstale":"2023-12-12T09:52:13.281819+0000","last_undegraded":"2023-12-12T09:52:13.281819+0000","last_fullsized":"2023-12-12T09:52:13.281819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:39.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104053+0000","last_change":"2023-12-12T09:52:07.014406+0000","last_active":"2023-12-12T09:52:12.104053+0000","last_peered":"2023-12-12T09:52:12.104053+0000","last_clean":"2023-12-12T09:52:12.104053+0000","last_became_active":"2023-12-12T09:52:07.014067+0000","last_became_peered":"2023-12-12T09:52:07.014067+0000","last_unstale":"2023-12-12T09:52:12.104053+0000","last_undegraded":"2023-12-12T09:52:12.104053+0000","last_fullsized":"2023-12-12T09:52:12.104053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075313+0000","last_change":"2023-12-12T09:52:08.025152+0000","last_active":"2023-12-12T09:52:12.075313+0000","last_peered":"2023-12-12T09:52:12.075313+0000","last_clean":"2023-12-12T09:52:12.075313+0000","last_became_active":"2023-12-12T09:52:08.024282+0000","last_became_peered":"2023-12-12T09:52:08.024282+0000","last_unstale":"2023-12-12T09:52:12.075313+0000","last_undegraded":"2023-12-12T09:52:12.075313+0000","last_fullsized":"2023-12-12T09:52:12.075313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:35.813548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281955+0000","last_change":"2023-12-12T09:52:12.079381+0000","last_active":"2023-12-12T09:52:13.281955+0000","last_peered":"2023-12-12T09:52:13.281955+0000","last_clean":"2023-12-12T09:52:13.281955+0000","last_became_active":"2023-12-12T09:52:12.078805+0000","last_became_peered":"2023-12-12T09:52:12.078805+0000","last_unstale":"2023-12-12T09:52:13.281955+0000","last_undegraded":"2023-12-12T09:52:13.281955+0000","last_fullsized":"2023-12-12T09:52:13.281955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:21:02.014413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050630+0000","last_change":"2023-12-12T09:52:08.020343+0000","last_active":"2023-12-12T09:52:12.050630+0000","last_peered":"2023-12-12T09:52:12.050630+0000","last_clean":"2023-12-12T09:52:12.050630+0000","last_became_active":"2023-12-12T09:52:08.019981+0000","last_became_peered":"2023-12-12T09:52:08.019981+0000","last_unstale":"2023-12-12T09:52:12.050630+0000","last_undegraded":"2023-12-12T09:52:12.050630+0000","last_fullsized":"2023-12-12T09:52:12.050630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104677+0000","last_change":"2023-12-12T09:52:07.011835+0000","last_active":"2023-12-12T09:52:12.104677+0000","last_peered":"2023-12-12T09:52:12.104677+0000","last_clean":"2023-12-12T09:52:12.104677+0000","last_became_active":"2023-12-12T09:52:07.011505+0000","last_became_peered":"2023-12-12T09:52:07.011505+0000","last_unstale":"2023-12-12T09:52:12.104677+0000","last_undegraded":"2023-12-12T09:52:12.104677+0000","last_fullsized":"2023-12-12T09:52:12.104677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285005+0000","last_change":"2023-12-12T09:52:12.080462+0000","last_active":"2023-12-12T09:52:13.285005+0000","last_peered":"2023-12-12T09:52:13.285005+0000","last_clean":"2023-12-12T09:52:13.285005+0000","last_became_active":"2023-12-12T09:52:12.080128+0000","last_became_peered":"2023-12-12T09:52:12.080128+0000","last_unstale":"2023-12-12T09:52:13.285005+0000","last_undegraded":"2023-12-12T09:52:13.285005+0000","last_fullsized":"2023-12-12T09:52:13.285005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054727+0000","last_change":"2023-12-12T09:52:08.028847+0000","last_active":"2023-12-12T09:52:12.054727+0000","last_peered":"2023-12-12T09:52:12.054727+0000","last_clean":"2023-12-12T09:52:12.054727+0000","last_became_active":"2023-12-12T09:52:08.028537+0000","last_became_peered":"2023-12-12T09:52:08.028537+0000","last_unstale":"2023-12-12T09:52:12.054727+0000","last_undegraded":"2023-12-12T09:52:12.054727+0000","last_fullsized":"2023-12-12T09:52:12.054727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068568+0000","last_change":"2023-12-12T09:52:06.992502+0000","last_active":"2023-12-12T09:52:12.068568+0000","last_peered":"2023-12-12T09:52:12.068568+0000","last_clean":"2023-12-12T09:52:12.068568+0000","last_became_active":"2023-12-12T09:52:06.992218+0000","last_became_peered":"2023-12-12T09:52:06.992218+0000","last_unstale":"2023-12-12T09:52:12.068568+0000","last_undegraded":"2023-12-12T09:52:12.068568+0000","last_fullsized":"2023-12-12T09:52:12.068568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284506+0000","last_change":"2023-12-12T09:52:12.083926+0000","last_active":"2023-12-12T09:52:13.284506+0000","last_peered":"2023-12-12T09:52:13.284506+0000","last_clean":"2023-12-12T09:52:13.284506+0000","last_became_active":"2023-12-12T09:52:12.083322+0000","last_became_peered":"2023-12-12T09:52:12.083322+0000","last_unstale":"2023-12-12T09:52:13.284506+0000","last_undegraded":"2023-12-12T09:52:13.284506+0000","last_fullsized":"2023-12-12T09:52:13.284506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:31.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059740+0000","last_change":"2023-12-12T09:52:08.021868+0000","last_active":"2023-12-12T09:52:12.059740+0000","last_peered":"2023-12-12T09:52:12.059740+0000","last_clean":"2023-12-12T09:52:12.059740+0000","last_became_active":"2023-12-12T09:52:08.020477+0000","last_became_peered":"2023-12-12T09:52:08.020477+0000","last_unstale":"2023-12-12T09:52:12.059740+0000","last_undegraded":"2023-12-12T09:52:12.059740+0000","last_fullsized":"2023-12-12T09:52:12.059740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055893+0000","last_change":"2023-12-12T09:52:07.005983+0000","last_active":"2023-12-12T09:52:12.055893+0000","last_peered":"2023-12-12T09:52:12.055893+0000","last_clean":"2023-12-12T09:52:12.055893+0000","last_became_active":"2023-12-12T09:52:07.005723+0000","last_became_peered":"2023-12-12T09:52:07.005723+0000","last_unstale":"2023-12-12T09:52:12.055893+0000","last_undegraded":"2023-12-12T09:52:12.055893+0000","last_fullsized":"2023-12-12T09:52:12.055893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282866+0000","last_change":"2023-12-12T09:52:12.110411+0000","last_active":"2023-12-12T09:52:13.282866+0000","last_peered":"2023-12-12T09:52:13.282866+0000","last_clean":"2023-12-12T09:52:13.282866+0000","last_became_active":"2023-12-12T09:52:12.110045+0000","last_became_peered":"2023-12-12T09:52:12.110045+0000","last_unstale":"2023-12-12T09:52:13.282866+0000","last_undegraded":"2023-12-12T09:52:13.282866+0000","last_fullsized":"2023-12-12T09:52:13.282866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:39.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.069034+0000","last_change":"2023-12-12T09:52:08.034974+0000","last_active":"2023-12-12T09:52:12.069034+0000","last_peered":"2023-12-12T09:52:12.069034+0000","last_clean":"2023-12-12T09:52:12.069034+0000","last_became_active":"2023-12-12T09:52:08.034169+0000","last_became_peered":"2023-12-12T09:52:08.034169+0000","last_unstale":"2023-12-12T09:52:12.069034+0000","last_undegraded":"2023-12-12T09:52:12.069034+0000","last_fullsized":"2023-12-12T09:52:12.069034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:47:41.840110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103698+0000","last_change":"2023-12-12T09:52:07.006046+0000","last_active":"2023-12-12T09:52:12.103698+0000","last_peered":"2023-12-12T09:52:12.103698+0000","last_clean":"2023-12-12T09:52:12.103698+0000","last_became_active":"2023-12-12T09:52:07.005742+0000","last_became_peered":"2023-12-12T09:52:07.005742+0000","last_unstale":"2023-12-12T09:52:12.103698+0000","last_undegraded":"2023-12-12T09:52:12.103698+0000","last_fullsized":"2023-12-12T09:52:12.103698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:31.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285141+0000","last_change":"2023-12-12T09:52:12.084173+0000","last_active":"2023-12-12T09:52:13.285141+0000","last_peered":"2023-12-12T09:52:13.285141+0000","last_clean":"2023-12-12T09:52:13.285141+0000","last_became_active":"2023-12-12T09:52:12.083324+0000","last_became_peered":"2023-12-12T09:52:12.083324+0000","last_unstale":"2023-12-12T09:52:13.285141+0000","last_undegraded":"2023-12-12T09:52:13.285141+0000","last_fullsized":"2023-12-12T09:52:13.285141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.071231+0000","last_change":"2023-12-12T09:52:08.022538+0000","last_active":"2023-12-12T09:52:12.071231+0000","last_peered":"2023-12-12T09:52:12.071231+0000","last_clean":"2023-12-12T09:52:12.071231+0000","last_became_active":"2023-12-12T09:52:08.022274+0000","last_became_peered":"2023-12-12T09:52:08.022274+0000","last_unstale":"2023-12-12T09:52:12.071231+0000","last_undegraded":"2023-12-12T09:52:12.071231+0000","last_fullsized":"2023-12-12T09:52:12.071231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053881+0000","last_change":"2023-12-12T09:52:07.012094+0000","last_active":"2023-12-12T09:52:12.053881+0000","last_peered":"2023-12-12T09:52:12.053881+0000","last_clean":"2023-12-12T09:52:12.053881+0000","last_became_active":"2023-12-12T09:52:07.011693+0000","last_became_peered":"2023-12-12T09:52:07.011693+0000","last_unstale":"2023-12-12T09:52:12.053881+0000","last_undegraded":"2023-12-12T09:52:12.053881+0000","last_fullsized":"2023-12-12T09:52:12.053881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:47.890555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281952+0000","last_change":"2023-12-12T09:52:12.111616+0000","last_active":"2023-12-12T09:52:13.281952+0000","last_peered":"2023-12-12T09:52:13.281952+0000","last_clean":"2023-12-12T09:52:13.281952+0000","last_became_active":"2023-12-12T09:52:12.110326+0000","last_became_peered":"2023-12-12T09:52:12.110326+0000","last_unstale":"2023-12-12T09:52:13.281952+0000","last_undegraded":"2023-12-12T09:52:13.281952+0000","last_fullsized":"2023-12-12T09:52:13.281952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:34:01.832456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.063408+0000","last_change":"2023-12-12T09:52:08.034520+0000","last_active":"2023-12-12T09:52:12.063408+0000","last_peered":"2023-12-12T09:52:12.063408+0000","last_clean":"2023-12-12T09:52:12.063408+0000","last_became_active":"2023-12-12T09:52:08.034232+0000","last_became_peered":"2023-12-12T09:52:08.034232+0000","last_unstale":"2023-12-12T09:52:12.063408+0000","last_undegraded":"2023-12-12T09:52:12.063408+0000","last_fullsized":"2023-12-12T09:52:12.063408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.760548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.057080+0000","last_change":"2023-12-12T09:52:07.004065+0000","last_active":"2023-12-12T09:52:12.057080+0000","last_peered":"2023-12-12T09:52:12.057080+0000","last_clean":"2023-12-12T09:52:12.057080+0000","last_became_active":"2023-12-12T09:52:07.003690+0000","last_became_peered":"2023-12-12T09:52:07.003690+0000","last_unstale":"2023-12-12T09:52:12.057080+0000","last_undegraded":"2023-12-12T09:52:12.057080+0000","last_fullsized":"2023-12-12T09:52:12.057080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:47.890555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285159+0000","last_change":"2023-12-12T09:52:12.108409+0000","last_active":"2023-12-12T09:52:13.285159+0000","last_peered":"2023-12-12T09:52:13.285159+0000","last_clean":"2023-12-12T09:52:13.285159+0000","last_became_active":"2023-12-12T09:52:12.107934+0000","last_became_peered":"2023-12-12T09:52:12.107934+0000","last_unstale":"2023-12-12T09:52:13.285159+0000","last_undegraded":"2023-12-12T09:52:13.285159+0000","last_fullsized":"2023-12-12T09:52:13.285159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:20.552141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065611+0000","last_change":"2023-12-12T09:52:08.031660+0000","last_active":"2023-12-12T09:52:12.065611+0000","last_peered":"2023-12-12T09:52:12.065611+0000","last_clean":"2023-12-12T09:52:12.065611+0000","last_became_active":"2023-12-12T09:52:08.030341+0000","last_became_peered":"2023-12-12T09:52:08.030341+0000","last_unstale":"2023-12-12T09:52:12.065611+0000","last_undegraded":"2023-12-12T09:52:12.065611+0000","last_fullsized":"2023-12-12T09:52:12.065611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104424+0000","last_change":"2023-12-12T09:52:07.014640+0000","last_active":"2023-12-12T09:52:12.104424+0000","last_peered":"2023-12-12T09:52:12.104424+0000","last_clean":"2023-12-12T09:52:12.104424+0000","last_became_active":"2023-12-12T09:52:07.014313+0000","last_became_peered":"2023-12-12T09:52:07.014313+0000","last_unstale":"2023-12-12T09:52:12.104424+0000","last_undegraded":"2023-12-12T09:52:12.104424+0000","last_fullsized":"2023-12-12T09:52:12.104424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:25.514763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285307+0000","last_change":"2023-12-12T09:52:12.111140+0000","last_active":"2023-12-12T09:52:13.285307+0000","last_peered":"2023-12-12T09:52:13.285307+0000","last_clean":"2023-12-12T09:52:13.285307+0000","last_became_active":"2023-12-12T09:52:12.110437+0000","last_became_peered":"2023-12-12T09:52:12.110437+0000","last_unstale":"2023-12-12T09:52:13.285307+0000","last_undegraded":"2023-12-12T09:52:13.285307+0000","last_fullsized":"2023-12-12T09:52:13.285307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075331+0000","last_change":"2023-12-12T09:52:08.031638+0000","last_active":"2023-12-12T09:52:12.075331+0000","last_peered":"2023-12-12T09:52:12.075331+0000","last_clean":"2023-12-12T09:52:12.075331+0000","last_became_active":"2023-12-12T09:52:08.031330+0000","last_became_peered":"2023-12-12T09:52:08.031330+0000","last_unstale":"2023-12-12T09:52:12.075331+0000","last_undegraded":"2023-12-12T09:52:12.075331+0000","last_fullsized":"2023-12-12T09:52:12.075331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:27.371886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065722+0000","last_change":"2023-12-12T09:52:06.996775+0000","last_active":"2023-12-12T09:52:12.065722+0000","last_peered":"2023-12-12T09:52:12.065722+0000","last_clean":"2023-12-12T09:52:12.065722+0000","last_became_active":"2023-12-12T09:52:06.995618+0000","last_became_peered":"2023-12-12T09:52:06.995618+0000","last_unstale":"2023-12-12T09:52:12.065722+0000","last_undegraded":"2023-12-12T09:52:12.065722+0000","last_fullsized":"2023-12-12T09:52:12.065722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:26:34.669025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282930+0000","last_change":"2023-12-12T09:52:12.085685+0000","last_active":"2023-12-12T09:52:13.282930+0000","last_peered":"2023-12-12T09:52:13.282930+0000","last_clean":"2023-12-12T09:52:13.282930+0000","last_became_active":"2023-12-12T09:52:12.085247+0000","last_became_peered":"2023-12-12T09:52:12.085247+0000","last_unstale":"2023-12-12T09:52:13.282930+0000","last_undegraded":"2023-12-12T09:52:13.282930+0000","last_fullsized":"2023-12-12T09:52:13.282930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:31.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070066+0000","last_change":"2023-12-12T09:52:08.020254+0000","last_active":"2023-12-12T09:52:12.070066+0000","last_peered":"2023-12-12T09:52:12.070066+0000","last_clean":"2023-12-12T09:52:12.070066+0000","last_became_active":"2023-12-12T09:52:08.019979+0000","last_became_peered":"2023-12-12T09:52:08.019979+0000","last_unstale":"2023-12-12T09:52:12.070066+0000","last_undegraded":"2023-12-12T09:52:12.070066+0000","last_fullsized":"2023-12-12T09:52:12.070066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:32.163069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074498+0000","last_change":"2023-12-12T09:52:06.997560+0000","last_active":"2023-12-12T09:52:12.074498+0000","last_peered":"2023-12-12T09:52:12.074498+0000","last_clean":"2023-12-12T09:52:12.074498+0000","last_became_active":"2023-12-12T09:52:06.996940+0000","last_became_peered":"2023-12-12T09:52:06.996940+0000","last_unstale":"2023-12-12T09:52:12.074498+0000","last_undegraded":"2023-12-12T09:52:12.074498+0000","last_fullsized":"2023-12-12T09:52:12.074498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:09.895493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285696+0000","last_change":"2023-12-12T09:52:12.077447+0000","last_active":"2023-12-12T09:52:13.285696+0000","last_peered":"2023-12-12T09:52:13.285696+0000","last_clean":"2023-12-12T09:52:13.285696+0000","last_became_active":"2023-12-12T09:52:12.076927+0000","last_became_peered":"2023-12-12T09:52:12.076927+0000","last_unstale":"2023-12-12T09:52:13.285696+0000","last_undegraded":"2023-12-12T09:52:13.285696+0000","last_fullsized":"2023-12-12T09:52:13.285696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:55.497789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.057627+0000","last_change":"2023-12-12T09:52:08.035111+0000","last_active":"2023-12-12T09:52:12.057627+0000","last_peered":"2023-12-12T09:52:12.057627+0000","last_clean":"2023-12-12T09:52:12.057627+0000","last_became_active":"2023-12-12T09:52:08.034823+0000","last_became_peered":"2023-12-12T09:52:08.034823+0000","last_unstale":"2023-12-12T09:52:12.057627+0000","last_undegraded":"2023-12-12T09:52:12.057627+0000","last_fullsized":"2023-12-12T09:52:12.057627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055046+0000","last_change":"2023-12-12T09:52:06.993403+0000","last_active":"2023-12-12T09:52:12.055046+0000","last_peered":"2023-12-12T09:52:12.055046+0000","last_clean":"2023-12-12T09:52:12.055046+0000","last_became_active":"2023-12-12T09:52:06.993116+0000","last_became_peered":"2023-12-12T09:52:06.993116+0000","last_unstale":"2023-12-12T09:52:12.055046+0000","last_undegraded":"2023-12-12T09:52:12.055046+0000","last_fullsized":"2023-12-12T09:52:12.055046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283254+0000","last_change":"2023-12-12T09:52:12.080184+0000","last_active":"2023-12-12T09:52:13.283254+0000","last_peered":"2023-12-12T09:52:13.283254+0000","last_clean":"2023-12-12T09:52:13.283254+0000","last_became_active":"2023-12-12T09:52:12.079804+0000","last_became_peered":"2023-12-12T09:52:12.079804+0000","last_unstale":"2023-12-12T09:52:13.283254+0000","last_undegraded":"2023-12-12T09:52:13.283254+0000","last_fullsized":"2023-12-12T09:52:13.283254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065376+0000","last_change":"2023-12-12T09:52:08.026908+0000","last_active":"2023-12-12T09:52:12.065376+0000","last_peered":"2023-12-12T09:52:12.065376+0000","last_clean":"2023-12-12T09:52:12.065376+0000","last_became_active":"2023-12-12T09:52:08.025912+0000","last_became_peered":"2023-12-12T09:52:08.025912+0000","last_unstale":"2023-12-12T09:52:12.065376+0000","last_undegraded":"2023-12-12T09:52:12.065376+0000","last_fullsized":"2023-12-12T09:52:12.065376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103688+0000","last_change":"2023-12-12T09:52:07.009290+0000","last_active":"2023-12-12T09:52:12.103688+0000","last_peered":"2023-12-12T09:52:12.103688+0000","last_clean":"2023-12-12T09:52:12.103688+0000","last_became_active":"2023-12-12T09:52:07.008896+0000","last_became_peered":"2023-12-12T09:52:07.008896+0000","last_unstale":"2023-12-12T09:52:12.103688+0000","last_undegraded":"2023-12-12T09:52:12.103688+0000","last_fullsized":"2023-12-12T09:52:12.103688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:21.191123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.290045+0000","last_change":"2023-12-12T09:52:12.115265+0000","last_active":"2023-12-12T09:52:13.290045+0000","last_peered":"2023-12-12T09:52:13.290045+0000","last_clean":"2023-12-12T09:52:13.290045+0000","last_became_active":"2023-12-12T09:52:12.114674+0000","last_became_peered":"2023-12-12T09:52:12.114674+0000","last_unstale":"2023-12-12T09:52:13.290045+0000","last_undegraded":"2023-12-12T09:52:13.290045+0000","last_fullsized":"2023-12-12T09:52:13.290045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:34:54.772153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060809+0000","last_change":"2023-12-12T09:52:08.028299+0000","last_active":"2023-12-12T09:52:12.060809+0000","last_peered":"2023-12-12T09:52:12.060809+0000","last_clean":"2023-12-12T09:52:12.060809+0000","last_became_active":"2023-12-12T09:52:08.028080+0000","last_became_peered":"2023-12-12T09:52:08.028080+0000","last_unstale":"2023-12-12T09:52:12.060809+0000","last_undegraded":"2023-12-12T09:52:12.060809+0000","last_fullsized":"2023-12-12T09:52:12.060809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074571+0000","last_change":"2023-12-12T09:52:06.996346+0000","last_active":"2023-12-12T09:52:12.074571+0000","last_peered":"2023-12-12T09:52:12.074571+0000","last_clean":"2023-12-12T09:52:12.074571+0000","last_became_active":"2023-12-12T09:52:06.995462+0000","last_became_peered":"2023-12-12T09:52:06.995462+0000","last_unstale":"2023-12-12T09:52:12.074571+0000","last_undegraded":"2023-12-12T09:52:12.074571+0000","last_fullsized":"2023-12-12T09:52:12.074571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:59.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103796+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:52:12.103796+0000","last_peered":"2023-12-12T09:52:12.103796+0000","last_clean":"2023-12-12T09:52:12.103796+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:52:12.103796+0000","last_undegraded":"2023-12-12T09:52:12.103796+0000","last_fullsized":"2023-12-12T09:52:12.103796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.055422+0000","last_change":"2023-12-12T09:52:12.109683+0000","last_active":"2023-12-12T09:52:13.055422+0000","last_peered":"2023-12-12T09:52:13.055422+0000","last_clean":"2023-12-12T09:52:13.055422+0000","last_became_active":"2023-12-12T09:52:12.108319+0000","last_became_peered":"2023-12-12T09:52:12.108319+0000","last_unstale":"2023-12-12T09:52:13.055422+0000","last_undegraded":"2023-12-12T09:52:13.055422+0000","last_fullsized":"2023-12-12T09:52:13.055422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:34:01.832456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104240+0000","last_change":"2023-12-12T09:52:08.033828+0000","last_active":"2023-12-12T09:52:12.104240+0000","last_peered":"2023-12-12T09:52:12.104240+0000","last_clean":"2023-12-12T09:52:12.104240+0000","last_became_active":"2023-12-12T09:52:08.030396+0000","last_became_peered":"2023-12-12T09:52:08.030396+0000","last_unstale":"2023-12-12T09:52:12.104240+0000","last_undegraded":"2023-12-12T09:52:12.104240+0000","last_fullsized":"2023-12-12T09:52:12.104240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:27.371886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060933+0000","last_change":"2023-12-12T09:52:06.998515+0000","last_active":"2023-12-12T09:52:12.060933+0000","last_peered":"2023-12-12T09:52:12.060933+0000","last_clean":"2023-12-12T09:52:12.060933+0000","last_became_active":"2023-12-12T09:52:06.998220+0000","last_became_peered":"2023-12-12T09:52:06.998220+0000","last_unstale":"2023-12-12T09:52:12.060933+0000","last_undegraded":"2023-12-12T09:52:12.060933+0000","last_fullsized":"2023-12-12T09:52:12.060933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:19.282948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.054359+0000","last_change":"2023-12-12T09:52:12.106813+0000","last_active":"2023-12-12T09:52:13.054359+0000","last_peered":"2023-12-12T09:52:13.054359+0000","last_clean":"2023-12-12T09:52:13.054359+0000","last_became_active":"2023-12-12T09:52:12.106334+0000","last_became_peered":"2023-12-12T09:52:12.106334+0000","last_unstale":"2023-12-12T09:52:13.054359+0000","last_undegraded":"2023-12-12T09:52:13.054359+0000","last_fullsized":"2023-12-12T09:52:13.054359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:39.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074947+0000","last_change":"2023-12-12T09:52:08.033424+0000","last_active":"2023-12-12T09:52:12.074947+0000","last_peered":"2023-12-12T09:52:12.074947+0000","last_clean":"2023-12-12T09:52:12.074947+0000","last_became_active":"2023-12-12T09:52:08.030055+0000","last_became_peered":"2023-12-12T09:52:08.030055+0000","last_unstale":"2023-12-12T09:52:12.074947+0000","last_undegraded":"2023-12-12T09:52:12.074947+0000","last_fullsized":"2023-12-12T09:52:12.074947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:32.450565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056464+0000","last_change":"2023-12-12T09:52:07.000843+0000","last_active":"2023-12-12T09:52:12.056464+0000","last_peered":"2023-12-12T09:52:12.056464+0000","last_clean":"2023-12-12T09:52:12.056464+0000","last_became_active":"2023-12-12T09:52:07.000476+0000","last_became_peered":"2023-12-12T09:52:07.000476+0000","last_unstale":"2023-12-12T09:52:12.056464+0000","last_undegraded":"2023-12-12T09:52:12.056464+0000","last_fullsized":"2023-12-12T09:52:12.056464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:59.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285509+0000","last_change":"2023-12-12T09:52:12.082760+0000","last_active":"2023-12-12T09:52:13.285509+0000","last_peered":"2023-12-12T09:52:13.285509+0000","last_clean":"2023-12-12T09:52:13.285509+0000","last_became_active":"2023-12-12T09:52:12.082269+0000","last_became_peered":"2023-12-12T09:52:12.082269+0000","last_unstale":"2023-12-12T09:52:13.285509+0000","last_undegraded":"2023-12-12T09:52:13.285509+0000","last_fullsized":"2023-12-12T09:52:13.285509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104860+0000","last_change":"2023-12-12T09:52:08.035636+0000","last_active":"2023-12-12T09:52:12.104860+0000","last_peered":"2023-12-12T09:52:12.104860+0000","last_clean":"2023-12-12T09:52:12.104860+0000","last_became_active":"2023-12-12T09:52:08.035306+0000","last_became_peered":"2023-12-12T09:52:08.035306+0000","last_unstale":"2023-12-12T09:52:12.104860+0000","last_undegraded":"2023-12-12T09:52:12.104860+0000","last_fullsized":"2023-12-12T09:52:12.104860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:34:50.708538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053859+0000","last_change":"2023-12-12T09:52:06.993300+0000","last_active":"2023-12-12T09:52:12.053859+0000","last_peered":"2023-12-12T09:52:12.053859+0000","last_clean":"2023-12-12T09:52:12.053859+0000","last_became_active":"2023-12-12T09:52:06.992970+0000","last_became_peered":"2023-12-12T09:52:06.992970+0000","last_unstale":"2023-12-12T09:52:12.053859+0000","last_undegraded":"2023-12-12T09:52:12.053859+0000","last_fullsized":"2023-12-12T09:52:12.053859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:21.191123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.057884+0000","last_change":"2023-12-12T09:52:12.107302+0000","last_active":"2023-12-12T09:52:13.057884+0000","last_peered":"2023-12-12T09:52:13.057884+0000","last_clean":"2023-12-12T09:52:13.057884+0000","last_became_active":"2023-12-12T09:52:12.106810+0000","last_became_peered":"2023-12-12T09:52:12.106810+0000","last_unstale":"2023-12-12T09:52:13.057884+0000","last_undegraded":"2023-12-12T09:52:13.057884+0000","last_fullsized":"2023-12-12T09:52:13.057884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:25:03.440488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104689+0000","last_change":"2023-12-12T09:52:08.039951+0000","last_active":"2023-12-12T09:52:12.104689+0000","last_peered":"2023-12-12T09:52:12.104689+0000","last_clean":"2023-12-12T09:52:12.104689+0000","last_became_active":"2023-12-12T09:52:08.039124+0000","last_became_peered":"2023-12-12T09:52:08.039124+0000","last_unstale":"2023-12-12T09:52:12.104689+0000","last_undegraded":"2023-12-12T09:52:12.104689+0000","last_fullsized":"2023-12-12T09:52:12.104689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:32.450565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066373+0000","last_change":"2023-12-12T09:52:06.998901+0000","last_active":"2023-12-12T09:52:12.066373+0000","last_peered":"2023-12-12T09:52:12.066373+0000","last_clean":"2023-12-12T09:52:12.066373+0000","last_became_active":"2023-12-12T09:52:06.998458+0000","last_became_peered":"2023-12-12T09:52:06.998458+0000","last_unstale":"2023-12-12T09:52:12.066373+0000","last_undegraded":"2023-12-12T09:52:12.066373+0000","last_fullsized":"2023-12-12T09:52:12.066373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281456+0000","last_change":"2023-12-12T09:52:12.080392+0000","last_active":"2023-12-12T09:52:13.281456+0000","last_peered":"2023-12-12T09:52:13.281456+0000","last_clean":"2023-12-12T09:52:13.281456+0000","last_became_active":"2023-12-12T09:52:12.079941+0000","last_became_peered":"2023-12-12T09:52:12.079941+0000","last_unstale":"2023-12-12T09:52:13.281456+0000","last_undegraded":"2023-12-12T09:52:13.281456+0000","last_fullsized":"2023-12-12T09:52:13.281456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:10.824163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103824+0000","last_change":"2023-12-12T09:52:08.032605+0000","last_active":"2023-12-12T09:52:12.103824+0000","last_peered":"2023-12-12T09:52:12.103824+0000","last_clean":"2023-12-12T09:52:12.103824+0000","last_became_active":"2023-12-12T09:52:08.032321+0000","last_became_peered":"2023-12-12T09:52:08.032321+0000","last_unstale":"2023-12-12T09:52:12.103824+0000","last_undegraded":"2023-12-12T09:52:12.103824+0000","last_fullsized":"2023-12-12T09:52:12.103824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053766+0000","last_change":"2023-12-12T09:52:06.993710+0000","last_active":"2023-12-12T09:52:12.053766+0000","last_peered":"2023-12-12T09:52:12.053766+0000","last_clean":"2023-12-12T09:52:12.053766+0000","last_became_active":"2023-12-12T09:52:06.993427+0000","last_became_peered":"2023-12-12T09:52:06.993427+0000","last_unstale":"2023-12-12T09:52:12.053766+0000","last_undegraded":"2023-12-12T09:52:12.053766+0000","last_fullsized":"2023-12-12T09:52:12.053766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:39.005895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.289994+0000","last_change":"2023-12-12T09:52:12.083094+0000","last_active":"2023-12-12T09:52:13.289994+0000","last_peered":"2023-12-12T09:52:13.289994+0000","last_clean":"2023-12-12T09:52:13.289994+0000","last_became_active":"2023-12-12T09:52:12.082050+0000","last_became_peered":"2023-12-12T09:52:12.082050+0000","last_unstale":"2023-12-12T09:52:13.289994+0000","last_undegraded":"2023-12-12T09:52:13.289994+0000","last_fullsized":"2023-12-12T09:52:13.289994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:51.971159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050599+0000","last_change":"2023-12-12T09:52:08.020100+0000","last_active":"2023-12-12T09:52:12.050599+0000","last_peered":"2023-12-12T09:52:12.050599+0000","last_clean":"2023-12-12T09:52:12.050599+0000","last_became_active":"2023-12-12T09:52:08.019776+0000","last_became_peered":"2023-12-12T09:52:08.019776+0000","last_unstale":"2023-12-12T09:52:12.050599+0000","last_undegraded":"2023-12-12T09:52:12.050599+0000","last_fullsized":"2023-12-12T09:52:12.050599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050514+0000","last_change":"2023-12-12T09:52:07.002330+0000","last_active":"2023-12-12T09:52:12.050514+0000","last_peered":"2023-12-12T09:52:12.050514+0000","last_clean":"2023-12-12T09:52:12.050514+0000","last_became_active":"2023-12-12T09:52:07.001838+0000","last_became_peered":"2023-12-12T09:52:07.001838+0000","last_unstale":"2023-12-12T09:52:12.050514+0000","last_undegraded":"2023-12-12T09:52:12.050514+0000","last_fullsized":"2023-12-12T09:52:12.050514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:44.065611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.287475+0000","last_change":"2023-12-12T09:52:12.111595+0000","last_active":"2023-12-12T09:52:13.287475+0000","last_peered":"2023-12-12T09:52:13.287475+0000","last_clean":"2023-12-12T09:52:13.287475+0000","last_became_active":"2023-12-12T09:52:12.111143+0000","last_became_peered":"2023-12-12T09:52:12.111143+0000","last_unstale":"2023-12-12T09:52:13.287475+0000","last_undegraded":"2023-12-12T09:52:13.287475+0000","last_fullsized":"2023-12-12T09:52:13.287475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:07:22.383529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104101+0000","last_change":"2023-12-12T09:52:08.036655+0000","last_active":"2023-12-12T09:52:12.104101+0000","last_peered":"2023-12-12T09:52:12.104101+0000","last_clean":"2023-12-12T09:52:12.104101+0000","last_became_active":"2023-12-12T09:52:08.036260+0000","last_became_peered":"2023-12-12T09:52:08.036260+0000","last_unstale":"2023-12-12T09:52:12.104101+0000","last_undegraded":"2023-12-12T09:52:12.104101+0000","last_fullsized":"2023-12-12T09:52:12.104101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:35.813548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074437+0000","last_change":"2023-12-12T09:52:06.999233+0000","last_active":"2023-12-12T09:52:12.074437+0000","last_peered":"2023-12-12T09:52:12.074437+0000","last_clean":"2023-12-12T09:52:12.074437+0000","last_became_active":"2023-12-12T09:52:06.998962+0000","last_became_peered":"2023-12-12T09:52:06.998962+0000","last_unstale":"2023-12-12T09:52:12.074437+0000","last_undegraded":"2023-12-12T09:52:12.074437+0000","last_fullsized":"2023-12-12T09:52:12.074437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:25.514763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285663+0000","last_change":"2023-12-12T09:52:12.106941+0000","last_active":"2023-12-12T09:52:13.285663+0000","last_peered":"2023-12-12T09:52:13.285663+0000","last_clean":"2023-12-12T09:52:13.285663+0000","last_became_active":"2023-12-12T09:52:12.106547+0000","last_became_peered":"2023-12-12T09:52:12.106547+0000","last_unstale":"2023-12-12T09:52:13.285663+0000","last_undegraded":"2023-12-12T09:52:13.285663+0000","last_fullsized":"2023-12-12T09:52:13.285663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:04:27.791759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070910+0000","last_change":"2023-12-12T09:52:08.029830+0000","last_active":"2023-12-12T09:52:12.070910+0000","last_peered":"2023-12-12T09:52:12.070910+0000","last_clean":"2023-12-12T09:52:12.070910+0000","last_became_active":"2023-12-12T09:52:08.029455+0000","last_became_peered":"2023-12-12T09:52:08.029455+0000","last_unstale":"2023-12-12T09:52:12.070910+0000","last_undegraded":"2023-12-12T09:52:12.070910+0000","last_fullsized":"2023-12-12T09:52:12.070910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068345+0000","last_change":"2023-12-12T09:52:07.000933+0000","last_active":"2023-12-12T09:52:12.068345+0000","last_peered":"2023-12-12T09:52:12.068345+0000","last_clean":"2023-12-12T09:52:12.068345+0000","last_became_active":"2023-12-12T09:52:07.000646+0000","last_became_peered":"2023-12-12T09:52:07.000646+0000","last_unstale":"2023-12-12T09:52:12.068345+0000","last_undegraded":"2023-12-12T09:52:12.068345+0000","last_fullsized":"2023-12-12T09:52:12.068345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:44.065611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.059459+0000","last_change":"2023-12-12T09:52:12.110657+0000","last_active":"2023-12-12T09:52:13.059459+0000","last_peered":"2023-12-12T09:52:13.059459+0000","last_clean":"2023-12-12T09:52:13.059459+0000","last_became_active":"2023-12-12T09:52:12.109951+0000","last_became_peered":"2023-12-12T09:52:12.109951+0000","last_unstale":"2023-12-12T09:52:13.059459+0000","last_undegraded":"2023-12-12T09:52:13.059459+0000","last_fullsized":"2023-12-12T09:52:13.059459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:20.552141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104135+0000","last_change":"2023-12-12T09:52:08.035067+0000","last_active":"2023-12-12T09:52:12.104135+0000","last_peered":"2023-12-12T09:52:12.104135+0000","last_clean":"2023-12-12T09:52:12.104135+0000","last_became_active":"2023-12-12T09:52:08.034803+0000","last_became_peered":"2023-12-12T09:52:08.034803+0000","last_unstale":"2023-12-12T09:52:12.104135+0000","last_undegraded":"2023-12-12T09:52:12.104135+0000","last_fullsized":"2023-12-12T09:52:12.104135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:22.978364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:27.255184+0000","last_change":"2023-12-12T09:52:06.998675+0000","last_active":"2023-12-12T09:52:27.255184+0000","last_peered":"2023-12-12T09:52:27.255184+0000","last_clean":"2023-12-12T09:52:27.255184+0000","last_became_active":"2023-12-12T09:52:06.998364+0000","last_became_peered":"2023-12-12T09:52:06.998364+0000","last_unstale":"2023-12-12T09:52:27.255184+0000","last_undegraded":"2023-12-12T09:52:27.255184+0000","last_fullsized":"2023-12-12T09:52:27.255184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3839,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283450+0000","last_change":"2023-12-12T09:52:12.086437+0000","last_active":"2023-12-12T09:52:13.283450+0000","last_peered":"2023-12-12T09:52:13.283450+0000","last_clean":"2023-12-12T09:52:13.283450+0000","last_became_active":"2023-12-12T09:52:12.083068+0000","last_became_peered":"2023-12-12T09:52:12.083068+0000","last_unstale":"2023-12-12T09:52:13.283450+0000","last_undegraded":"2023-12-12T09:52:13.283450+0000","last_fullsized":"2023-12-12T09:52:13.283450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066756+0000","last_change":"2023-12-12T09:52:08.030198+0000","last_active":"2023-12-12T09:52:12.066756+0000","last_peered":"2023-12-12T09:52:12.066756+0000","last_clean":"2023-12-12T09:52:12.066756+0000","last_became_active":"2023-12-12T09:52:08.027733+0000","last_became_peered":"2023-12-12T09:52:08.027733+0000","last_unstale":"2023-12-12T09:52:12.066756+0000","last_undegraded":"2023-12-12T09:52:12.066756+0000","last_fullsized":"2023-12-12T09:52:12.066756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:08.788149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056359+0000","last_change":"2023-12-12T09:52:07.004177+0000","last_active":"2023-12-12T09:52:12.056359+0000","last_peered":"2023-12-12T09:52:12.056359+0000","last_clean":"2023-12-12T09:52:12.056359+0000","last_became_active":"2023-12-12T09:52:07.003836+0000","last_became_peered":"2023-12-12T09:52:07.003836+0000","last_unstale":"2023-12-12T09:52:12.056359+0000","last_undegraded":"2023-12-12T09:52:12.056359+0000","last_fullsized":"2023-12-12T09:52:12.056359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284483+0000","last_change":"2023-12-12T09:52:12.081391+0000","last_active":"2023-12-12T09:52:13.284483+0000","last_peered":"2023-12-12T09:52:13.284483+0000","last_clean":"2023-12-12T09:52:13.284483+0000","last_became_active":"2023-12-12T09:52:12.080949+0000","last_became_peered":"2023-12-12T09:52:12.080949+0000","last_unstale":"2023-12-12T09:52:13.284483+0000","last_undegraded":"2023-12-12T09:52:13.284483+0000","last_fullsized":"2023-12-12T09:52:13.284483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:39.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070164+0000","last_change":"2023-12-12T09:52:08.023308+0000","last_active":"2023-12-12T09:52:12.070164+0000","last_peered":"2023-12-12T09:52:12.070164+0000","last_clean":"2023-12-12T09:52:12.070164+0000","last_became_active":"2023-12-12T09:52:08.023048+0000","last_became_peered":"2023-12-12T09:52:08.023048+0000","last_unstale":"2023-12-12T09:52:12.070164+0000","last_undegraded":"2023-12-12T09:52:12.070164+0000","last_fullsized":"2023-12-12T09:52:12.070164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075599+0000","last_change":"2023-12-12T09:52:07.001746+0000","last_active":"2023-12-12T09:52:12.075599+0000","last_peered":"2023-12-12T09:52:12.075599+0000","last_clean":"2023-12-12T09:52:12.075599+0000","last_became_active":"2023-12-12T09:52:07.001424+0000","last_became_peered":"2023-12-12T09:52:07.001424+0000","last_unstale":"2023-12-12T09:52:12.075599+0000","last_undegraded":"2023-12-12T09:52:12.075599+0000","last_fullsized":"2023-12-12T09:52:12.075599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:54.297647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282877+0000","last_change":"2023-12-12T09:52:12.077880+0000","last_active":"2023-12-12T09:52:13.282877+0000","last_peered":"2023-12-12T09:52:13.282877+0000","last_clean":"2023-12-12T09:52:13.282877+0000","last_became_active":"2023-12-12T09:52:12.077365+0000","last_became_peered":"2023-12-12T09:52:12.077365+0000","last_unstale":"2023-12-12T09:52:13.282877+0000","last_undegraded":"2023-12-12T09:52:13.282877+0000","last_fullsized":"2023-12-12T09:52:13.282877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:43:05.307563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075525+0000","last_change":"2023-12-12T09:52:08.029359+0000","last_active":"2023-12-12T09:52:12.075525+0000","last_peered":"2023-12-12T09:52:12.075525+0000","last_clean":"2023-12-12T09:52:12.075525+0000","last_became_active":"2023-12-12T09:52:08.028997+0000","last_became_peered":"2023-12-12T09:52:08.028997+0000","last_unstale":"2023-12-12T09:52:12.075525+0000","last_undegraded":"2023-12-12T09:52:12.075525+0000","last_fullsized":"2023-12-12T09:52:12.075525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:22.978364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.052142+0000","last_change":"2023-12-12T09:52:07.006953+0000","last_active":"2023-12-12T09:52:12.052142+0000","last_peered":"2023-12-12T09:52:12.052142+0000","last_clean":"2023-12-12T09:52:12.052142+0000","last_became_active":"2023-12-12T09:52:07.006666+0000","last_became_peered":"2023-12-12T09:52:07.006666+0000","last_unstale":"2023-12-12T09:52:12.052142+0000","last_undegraded":"2023-12-12T09:52:12.052142+0000","last_fullsized":"2023-12-12T09:52:12.052142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:26:34.669025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283756+0000","last_change":"2023-12-12T09:52:12.104433+0000","last_active":"2023-12-12T09:52:13.283756+0000","last_peered":"2023-12-12T09:52:13.283756+0000","last_clean":"2023-12-12T09:52:13.283756+0000","last_became_active":"2023-12-12T09:52:12.104126+0000","last_became_peered":"2023-12-12T09:52:12.104126+0000","last_unstale":"2023-12-12T09:52:13.283756+0000","last_undegraded":"2023-12-12T09:52:13.283756+0000","last_fullsized":"2023-12-12T09:52:13.283756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:25:03.440488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.289874+0000","last_change":"2023-12-12T09:52:12.111376+0000","last_active":"2023-12-12T09:52:13.289874+0000","last_peered":"2023-12-12T09:52:13.289874+0000","last_clean":"2023-12-12T09:52:13.289874+0000","last_became_active":"2023-12-12T09:52:12.110863+0000","last_became_peered":"2023-12-12T09:52:12.110863+0000","last_unstale":"2023-12-12T09:52:13.289874+0000","last_undegraded":"2023-12-12T09:52:13.289874+0000","last_fullsized":"2023-12-12T09:52:13.289874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:55.497789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055671+0000","last_change":"2023-12-12T09:52:08.024511+0000","last_active":"2023-12-12T09:52:12.055671+0000","last_peered":"2023-12-12T09:52:12.055671+0000","last_clean":"2023-12-12T09:52:12.055671+0000","last_became_active":"2023-12-12T09:52:08.024258+0000","last_became_peered":"2023-12-12T09:52:08.024258+0000","last_unstale":"2023-12-12T09:52:12.055671+0000","last_undegraded":"2023-12-12T09:52:12.055671+0000","last_fullsized":"2023-12-12T09:52:12.055671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074250+0000","last_change":"2023-12-12T09:52:07.000641+0000","last_active":"2023-12-12T09:52:12.074250+0000","last_peered":"2023-12-12T09:52:12.074250+0000","last_clean":"2023-12-12T09:52:12.074250+0000","last_became_active":"2023-12-12T09:52:07.000245+0000","last_became_peered":"2023-12-12T09:52:07.000245+0000","last_unstale":"2023-12-12T09:52:12.074250+0000","last_undegraded":"2023-12-12T09:52:12.074250+0000","last_fullsized":"2023-12-12T09:52:12.074250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:46.546348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.056067+0000","last_change":"2023-12-12T09:52:12.078229+0000","last_active":"2023-12-12T09:52:13.056067+0000","last_peered":"2023-12-12T09:52:13.056067+0000","last_clean":"2023-12-12T09:52:13.056067+0000","last_became_active":"2023-12-12T09:52:12.077764+0000","last_became_peered":"2023-12-12T09:52:12.077764+0000","last_unstale":"2023-12-12T09:52:13.056067+0000","last_undegraded":"2023-12-12T09:52:13.056067+0000","last_fullsized":"2023-12-12T09:52:13.056067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:21:02.014413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059639+0000","last_change":"2023-12-12T09:52:08.028827+0000","last_active":"2023-12-12T09:52:12.059639+0000","last_peered":"2023-12-12T09:52:12.059639+0000","last_clean":"2023-12-12T09:52:12.059639+0000","last_became_active":"2023-12-12T09:52:08.024860+0000","last_became_peered":"2023-12-12T09:52:08.024860+0000","last_unstale":"2023-12-12T09:52:12.059639+0000","last_undegraded":"2023-12-12T09:52:12.059639+0000","last_fullsized":"2023-12-12T09:52:12.059639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054975+0000","last_change":"2023-12-12T09:52:06.999381+0000","last_active":"2023-12-12T09:52:12.054975+0000","last_peered":"2023-12-12T09:52:12.054975+0000","last_clean":"2023-12-12T09:52:12.054975+0000","last_became_active":"2023-12-12T09:52:06.999099+0000","last_became_peered":"2023-12-12T09:52:06.999099+0000","last_unstale":"2023-12-12T09:52:12.054975+0000","last_undegraded":"2023-12-12T09:52:12.054975+0000","last_fullsized":"2023-12-12T09:52:12.054975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050899+0000","last_change":"2023-12-12T09:52:07.003879+0000","last_active":"2023-12-12T09:52:12.050899+0000","last_peered":"2023-12-12T09:52:12.050899+0000","last_clean":"2023-12-12T09:52:12.050899+0000","last_became_active":"2023-12-12T09:52:07.002760+0000","last_became_peered":"2023-12-12T09:52:07.002760+0000","last_unstale":"2023-12-12T09:52:12.050899+0000","last_undegraded":"2023-12-12T09:52:12.050899+0000","last_fullsized":"2023-12-12T09:52:12.050899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:07.777932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068247+0000","last_change":"2023-12-12T09:52:08.026740+0000","last_active":"2023-12-12T09:52:12.068247+0000","last_peered":"2023-12-12T09:52:12.068247+0000","last_clean":"2023-12-12T09:52:12.068247+0000","last_became_active":"2023-12-12T09:52:08.024698+0000","last_became_peered":"2023-12-12T09:52:08.024698+0000","last_unstale":"2023-12-12T09:52:12.068247+0000","last_undegraded":"2023-12-12T09:52:12.068247+0000","last_fullsized":"2023-12-12T09:52:12.068247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284952+0000","last_change":"2023-12-12T09:52:12.078748+0000","last_active":"2023-12-12T09:52:13.284952+0000","last_peered":"2023-12-12T09:52:13.284952+0000","last_clean":"2023-12-12T09:52:13.284952+0000","last_became_active":"2023-12-12T09:52:12.077889+0000","last_became_peered":"2023-12-12T09:52:12.077889+0000","last_unstale":"2023-12-12T09:52:13.284952+0000","last_undegraded":"2023-12-12T09:52:13.284952+0000","last_fullsized":"2023-12-12T09:52:13.284952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:05.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054823+0000","last_change":"2023-12-12T09:52:07.008787+0000","last_active":"2023-12-12T09:52:12.054823+0000","last_peered":"2023-12-12T09:52:12.054823+0000","last_clean":"2023-12-12T09:52:12.054823+0000","last_became_active":"2023-12-12T09:52:07.008455+0000","last_became_peered":"2023-12-12T09:52:07.008455+0000","last_unstale":"2023-12-12T09:52:12.054823+0000","last_undegraded":"2023-12-12T09:52:12.054823+0000","last_fullsized":"2023-12-12T09:52:12.054823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.061074+0000","last_change":"2023-12-12T09:52:08.022255+0000","last_active":"2023-12-12T09:52:12.061074+0000","last_peered":"2023-12-12T09:52:12.061074+0000","last_clean":"2023-12-12T09:52:12.061074+0000","last_became_active":"2023-12-12T09:52:08.020240+0000","last_became_peered":"2023-12-12T09:52:08.020240+0000","last_unstale":"2023-12-12T09:52:12.061074+0000","last_undegraded":"2023-12-12T09:52:12.061074+0000","last_fullsized":"2023-12-12T09:52:12.061074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:22.201340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.286721+0000","last_change":"2023-12-12T09:52:12.081878+0000","last_active":"2023-12-12T09:52:13.286721+0000","last_peered":"2023-12-12T09:52:13.286721+0000","last_clean":"2023-12-12T09:52:13.286721+0000","last_became_active":"2023-12-12T09:52:12.081502+0000","last_became_peered":"2023-12-12T09:52:12.081502+0000","last_unstale":"2023-12-12T09:52:13.286721+0000","last_undegraded":"2023-12-12T09:52:13.286721+0000","last_fullsized":"2023-12-12T09:52:13.286721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:18:41.559574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050317+0000","last_change":"2023-12-12T09:52:08.021237+0000","last_active":"2023-12-12T09:52:12.050317+0000","last_peered":"2023-12-12T09:52:12.050317+0000","last_clean":"2023-12-12T09:52:12.050317+0000","last_became_active":"2023-12-12T09:52:08.019519+0000","last_became_peered":"2023-12-12T09:52:08.019519+0000","last_unstale":"2023-12-12T09:52:12.050317+0000","last_undegraded":"2023-12-12T09:52:12.050317+0000","last_fullsized":"2023-12-12T09:52:12.050317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:01.082803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054863+0000","last_change":"2023-12-12T09:52:08.019363+0000","last_active":"2023-12-12T09:52:12.054863+0000","last_peered":"2023-12-12T09:52:12.054863+0000","last_clean":"2023-12-12T09:52:12.054863+0000","last_became_active":"2023-12-12T09:52:08.012574+0000","last_became_peered":"2023-12-12T09:52:08.012574+0000","last_unstale":"2023-12-12T09:52:12.054863+0000","last_undegraded":"2023-12-12T09:52:12.054863+0000","last_fullsized":"2023-12-12T09:52:12.054863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:08.788149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074382+0000","last_change":"2023-12-12T09:52:08.030165+0000","last_active":"2023-12-12T09:52:12.074382+0000","last_peered":"2023-12-12T09:52:12.074382+0000","last_clean":"2023-12-12T09:52:12.074382+0000","last_became_active":"2023-12-12T09:52:08.029626+0000","last_became_peered":"2023-12-12T09:52:08.029626+0000","last_unstale":"2023-12-12T09:52:12.074382+0000","last_undegraded":"2023-12-12T09:52:12.074382+0000","last_fullsized":"2023-12-12T09:52:12.074382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068155+0000","last_change":"2023-12-12T09:52:08.040599+0000","last_active":"2023-12-12T09:52:12.068155+0000","last_peered":"2023-12-12T09:52:12.068155+0000","last_clean":"2023-12-12T09:52:12.068155+0000","last_became_active":"2023-12-12T09:52:08.039875+0000","last_became_peered":"2023-12-12T09:52:08.039875+0000","last_unstale":"2023-12-12T09:52:12.068155+0000","last_undegraded":"2023-12-12T09:52:12.068155+0000","last_fullsized":"2023-12-12T09:52:12.068155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060010+0000","last_change":"2023-12-12T09:52:08.032398+0000","last_active":"2023-12-12T09:52:12.060010+0000","last_peered":"2023-12-12T09:52:12.060010+0000","last_clean":"2023-12-12T09:52:12.060010+0000","last_became_active":"2023-12-12T09:52:08.031194+0000","last_became_peered":"2023-12-12T09:52:08.031194+0000","last_unstale":"2023-12-12T09:52:12.060010+0000","last_undegraded":"2023-12-12T09:52:12.060010+0000","last_fullsized":"2023-12-12T09:52:12.060010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056859+0000","last_change":"2023-12-12T09:52:08.023831+0000","last_active":"2023-12-12T09:52:12.056859+0000","last_peered":"2023-12-12T09:52:12.056859+0000","last_clean":"2023-12-12T09:52:12.056859+0000","last_became_active":"2023-12-12T09:52:08.023532+0000","last_became_peered":"2023-12-12T09:52:08.023532+0000","last_unstale":"2023-12-12T09:52:12.056859+0000","last_undegraded":"2023-12-12T09:52:12.056859+0000","last_fullsized":"2023-12-12T09:52:12.056859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054670+0000","last_change":"2023-12-12T09:52:08.028457+0000","last_active":"2023-12-12T09:52:12.054670+0000","last_peered":"2023-12-12T09:52:12.054670+0000","last_clean":"2023-12-12T09:52:12.054670+0000","last_became_active":"2023-12-12T09:52:08.028172+0000","last_became_peered":"2023-12-12T09:52:08.028172+0000","last_unstale":"2023-12-12T09:52:12.054670+0000","last_undegraded":"2023-12-12T09:52:12.054670+0000","last_fullsized":"2023-12-12T09:52:12.054670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.063395+0000","last_change":"2023-12-12T09:52:08.033841+0000","last_active":"2023-12-12T09:52:12.063395+0000","last_peered":"2023-12-12T09:52:12.063395+0000","last_clean":"2023-12-12T09:52:12.063395+0000","last_became_active":"2023-12-12T09:52:08.030880+0000","last_became_peered":"2023-12-12T09:52:08.030880+0000","last_unstale":"2023-12-12T09:52:12.063395+0000","last_undegraded":"2023-12-12T09:52:12.063395+0000","last_fullsized":"2023-12-12T09:52:12.063395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074138+0000","last_change":"2023-12-12T09:52:08.032051+0000","last_active":"2023-12-12T09:52:12.074138+0000","last_peered":"2023-12-12T09:52:12.074138+0000","last_clean":"2023-12-12T09:52:12.074138+0000","last_became_active":"2023-12-12T09:52:08.031785+0000","last_became_peered":"2023-12-12T09:52:08.031785+0000","last_unstale":"2023-12-12T09:52:12.074138+0000","last_undegraded":"2023-12-12T09:52:12.074138+0000","last_fullsized":"2023-12-12T09:52:12.074138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054886+0000","last_change":"2023-12-12T09:52:08.031522+0000","last_active":"2023-12-12T09:52:12.054886+0000","last_peered":"2023-12-12T09:52:12.054886+0000","last_clean":"2023-12-12T09:52:12.054886+0000","last_became_active":"2023-12-12T09:52:08.031262+0000","last_became_peered":"2023-12-12T09:52:08.031262+0000","last_unstale":"2023-12-12T09:52:12.054886+0000","last_undegraded":"2023-12-12T09:52:12.054886+0000","last_fullsized":"2023-12-12T09:52:12.054886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074154+0000","last_change":"2023-12-12T09:52:08.024395+0000","last_active":"2023-12-12T09:52:12.074154+0000","last_peered":"2023-12-12T09:52:12.074154+0000","last_clean":"2023-12-12T09:52:12.074154+0000","last_became_active":"2023-12-12T09:52:08.024105+0000","last_became_peered":"2023-12-12T09:52:08.024105+0000","last_unstale":"2023-12-12T09:52:12.074154+0000","last_undegraded":"2023-12-12T09:52:12.074154+0000","last_fullsized":"2023-12-12T09:52:12.074154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.760548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.069490+0000","last_change":"2023-12-12T09:52:08.037425+0000","last_active":"2023-12-12T09:52:12.069490+0000","last_peered":"2023-12-12T09:52:12.069490+0000","last_clean":"2023-12-12T09:52:12.069490+0000","last_became_active":"2023-12-12T09:52:08.035756+0000","last_became_peered":"2023-12-12T09:52:08.035756+0000","last_unstale":"2023-12-12T09:52:12.069490+0000","last_undegraded":"2023-12-12T09:52:12.069490+0000","last_fullsized":"2023-12-12T09:52:12.069490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:01.082803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050903+0000","last_change":"2023-12-12T09:52:08.031988+0000","last_active":"2023-12-12T09:52:12.050903+0000","last_peered":"2023-12-12T09:52:12.050903+0000","last_clean":"2023-12-12T09:52:12.050903+0000","last_became_active":"2023-12-12T09:52:08.031525+0000","last_became_peered":"2023-12-12T09:52:08.031525+0000","last_unstale":"2023-12-12T09:52:12.050903+0000","last_undegraded":"2023-12-12T09:52:12.050903+0000","last_fullsized":"2023-12-12T09:52:12.050903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104392+0000","last_change":"2023-12-12T09:52:08.034992+0000","last_active":"2023-12-12T09:52:12.104392+0000","last_peered":"2023-12-12T09:52:12.104392+0000","last_clean":"2023-12-12T09:52:12.104392+0000","last_became_active":"2023-12-12T09:52:08.034424+0000","last_became_peered":"2023-12-12T09:52:08.034424+0000","last_unstale":"2023-12-12T09:52:12.104392+0000","last_undegraded":"2023-12-12T09:52:12.104392+0000","last_fullsized":"2023-12-12T09:52:12.104392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:24.493525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053756+0000","last_change":"2023-12-12T09:52:08.021603+0000","last_active":"2023-12-12T09:52:12.053756+0000","last_peered":"2023-12-12T09:52:12.053756+0000","last_clean":"2023-12-12T09:52:12.053756+0000","last_became_active":"2023-12-12T09:52:08.021253+0000","last_became_peered":"2023-12-12T09:52:08.021253+0000","last_unstale":"2023-12-12T09:52:12.053756+0000","last_undegraded":"2023-12-12T09:52:12.053756+0000","last_fullsized":"2023-12-12T09:52:12.053756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:45.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075822+0000","last_change":"2023-12-12T09:52:08.036026+0000","last_active":"2023-12-12T09:52:12.075822+0000","last_peered":"2023-12-12T09:52:12.075822+0000","last_clean":"2023-12-12T09:52:12.075822+0000","last_became_active":"2023-12-12T09:52:08.035390+0000","last_became_peered":"2023-12-12T09:52:08.035390+0000","last_unstale":"2023-12-12T09:52:12.075822+0000","last_undegraded":"2023-12-12T09:52:12.075822+0000","last_fullsized":"2023-12-12T09:52:12.075822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:51.434174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.073460+0000","last_change":"2023-12-12T09:52:08.013438+0000","last_active":"2023-12-12T09:52:12.073460+0000","last_peered":"2023-12-12T09:52:12.073460+0000","last_clean":"2023-12-12T09:52:12.073460+0000","last_became_active":"2023-12-12T09:52:08.013002+0000","last_became_peered":"2023-12-12T09:52:08.013002+0000","last_unstale":"2023-12-12T09:52:12.073460+0000","last_undegraded":"2023-12-12T09:52:12.073460+0000","last_fullsized":"2023-12-12T09:52:12.073460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055788+0000","last_change":"2023-12-12T09:52:08.032162+0000","last_active":"2023-12-12T09:52:12.055788+0000","last_peered":"2023-12-12T09:52:12.055788+0000","last_clean":"2023-12-12T09:52:12.055788+0000","last_became_active":"2023-12-12T09:52:08.031735+0000","last_became_peered":"2023-12-12T09:52:08.031735+0000","last_unstale":"2023-12-12T09:52:12.055788+0000","last_undegraded":"2023-12-12T09:52:12.055788+0000","last_fullsized":"2023-12-12T09:52:12.055788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070823+0000","last_change":"2023-12-12T09:52:08.024576+0000","last_active":"2023-12-12T09:52:12.070823+0000","last_peered":"2023-12-12T09:52:12.070823+0000","last_clean":"2023-12-12T09:52:12.070823+0000","last_became_active":"2023-12-12T09:52:08.024277+0000","last_became_peered":"2023-12-12T09:52:08.024277+0000","last_unstale":"2023-12-12T09:52:12.070823+0000","last_undegraded":"2023-12-12T09:52:12.070823+0000","last_fullsized":"2023-12-12T09:52:12.070823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:48.900772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054635+0000","last_change":"2023-12-12T09:52:08.029262+0000","last_active":"2023-12-12T09:52:12.054635+0000","last_peered":"2023-12-12T09:52:12.054635+0000","last_clean":"2023-12-12T09:52:12.054635+0000","last_became_active":"2023-12-12T09:52:08.028949+0000","last_became_peered":"2023-12-12T09:52:08.028949+0000","last_unstale":"2023-12-12T09:52:12.054635+0000","last_undegraded":"2023-12-12T09:52:12.054635+0000","last_fullsized":"2023-12-12T09:52:12.054635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066542+0000","last_change":"2023-12-12T09:52:08.024637+0000","last_active":"2023-12-12T09:52:12.066542+0000","last_peered":"2023-12-12T09:52:12.066542+0000","last_clean":"2023-12-12T09:52:12.066542+0000","last_became_active":"2023-12-12T09:52:08.023754+0000","last_became_peered":"2023-12-12T09:52:08.023754+0000","last_unstale":"2023-12-12T09:52:12.066542+0000","last_undegraded":"2023-12-12T09:52:12.066542+0000","last_fullsized":"2023-12-12T09:52:12.066542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:45.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055913+0000","last_change":"2023-12-12T09:52:08.024613+0000","last_active":"2023-12-12T09:52:12.055913+0000","last_peered":"2023-12-12T09:52:12.055913+0000","last_clean":"2023-12-12T09:52:12.055913+0000","last_became_active":"2023-12-12T09:52:08.024257+0000","last_became_peered":"2023-12-12T09:52:08.024257+0000","last_unstale":"2023-12-12T09:52:12.055913+0000","last_undegraded":"2023-12-12T09:52:12.055913+0000","last_fullsized":"2023-12-12T09:52:12.055913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054417+0000","last_change":"2023-12-12T09:52:08.023097+0000","last_active":"2023-12-12T09:52:12.054417+0000","last_peered":"2023-12-12T09:52:12.054417+0000","last_clean":"2023-12-12T09:52:12.054417+0000","last_became_active":"2023-12-12T09:52:08.022866+0000","last_became_peered":"2023-12-12T09:52:08.022866+0000","last_unstale":"2023-12-12T09:52:12.054417+0000","last_undegraded":"2023-12-12T09:52:12.054417+0000","last_fullsized":"2023-12-12T09:52:12.054417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104813+0000","last_change":"2023-12-12T09:52:08.039672+0000","last_active":"2023-12-12T09:52:12.104813+0000","last_peered":"2023-12-12T09:52:12.104813+0000","last_clean":"2023-12-12T09:52:12.104813+0000","last_became_active":"2023-12-12T09:52:08.039145+0000","last_became_peered":"2023-12-12T09:52:08.039145+0000","last_unstale":"2023-12-12T09:52:12.104813+0000","last_undegraded":"2023-12-12T09:52:12.104813+0000","last_fullsized":"2023-12-12T09:52:12.104813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:47.907544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059506+0000","last_change":"2023-12-12T09:52:08.030292+0000","last_active":"2023-12-12T09:52:12.059506+0000","last_peered":"2023-12-12T09:52:12.059506+0000","last_clean":"2023-12-12T09:52:12.059506+0000","last_became_active":"2023-12-12T09:52:08.029117+0000","last_became_peered":"2023-12-12T09:52:08.029117+0000","last_unstale":"2023-12-12T09:52:12.059506+0000","last_undegraded":"2023-12-12T09:52:12.059506+0000","last_fullsized":"2023-12-12T09:52:12.059506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103702+0000","last_change":"2023-12-12T09:52:08.040587+0000","last_active":"2023-12-12T09:52:12.103702+0000","last_peered":"2023-12-12T09:52:12.103702+0000","last_clean":"2023-12-12T09:52:12.103702+0000","last_became_active":"2023-12-12T09:52:08.040204+0000","last_became_peered":"2023-12-12T09:52:08.040204+0000","last_unstale":"2023-12-12T09:52:12.103702+0000","last_undegraded":"2023-12-12T09:52:12.103702+0000","last_fullsized":"2023-12-12T09:52:12.103702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.073636+0000","last_change":"2023-12-12T09:52:08.015126+0000","last_active":"2023-12-12T09:52:12.073636+0000","last_peered":"2023-12-12T09:52:12.073636+0000","last_clean":"2023-12-12T09:52:12.073636+0000","last_became_active":"2023-12-12T09:52:08.014817+0000","last_became_peered":"2023-12-12T09:52:08.014817+0000","last_unstale":"2023-12-12T09:52:12.073636+0000","last_undegraded":"2023-12-12T09:52:12.073636+0000","last_fullsized":"2023-12-12T09:52:12.073636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103966+0000","last_change":"2023-12-12T09:52:08.034136+0000","last_active":"2023-12-12T09:52:12.103966+0000","last_peered":"2023-12-12T09:52:12.103966+0000","last_clean":"2023-12-12T09:52:12.103966+0000","last_became_active":"2023-12-12T09:52:08.033810+0000","last_became_peered":"2023-12-12T09:52:08.033810+0000","last_unstale":"2023-12-12T09:52:12.103966+0000","last_undegraded":"2023-12-12T09:52:12.103966+0000","last_fullsized":"2023-12-12T09:52:12.103966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.950798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074787+0000","last_change":"2023-12-12T09:52:08.032774+0000","last_active":"2023-12-12T09:52:12.074787+0000","last_peered":"2023-12-12T09:52:12.074787+0000","last_clean":"2023-12-12T09:52:12.074787+0000","last_became_active":"2023-12-12T09:52:08.030758+0000","last_became_peered":"2023-12-12T09:52:08.030758+0000","last_unstale":"2023-12-12T09:52:12.074787+0000","last_undegraded":"2023-12-12T09:52:12.074787+0000","last_fullsized":"2023-12-12T09:52:12.074787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065632+0000","last_change":"2023-12-12T09:52:08.032385+0000","last_active":"2023-12-12T09:52:12.065632+0000","last_peered":"2023-12-12T09:52:12.065632+0000","last_clean":"2023-12-12T09:52:12.065632+0000","last_became_active":"2023-12-12T09:52:08.032067+0000","last_became_peered":"2023-12-12T09:52:08.032067+0000","last_unstale":"2023-12-12T09:52:12.065632+0000","last_undegraded":"2023-12-12T09:52:12.065632+0000","last_fullsized":"2023-12-12T09:52:12.065632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056995+0000","last_change":"2023-12-12T09:52:08.028105+0000","last_active":"2023-12-12T09:52:12.056995+0000","last_peered":"2023-12-12T09:52:12.056995+0000","last_clean":"2023-12-12T09:52:12.056995+0000","last_became_active":"2023-12-12T09:52:08.027786+0000","last_became_peered":"2023-12-12T09:52:08.027786+0000","last_unstale":"2023-12-12T09:52:12.056995+0000","last_undegraded":"2023-12-12T09:52:12.056995+0000","last_fullsized":"2023-12-12T09:52:12.056995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104765+0000","last_change":"2023-12-12T09:52:08.034061+0000","last_active":"2023-12-12T09:52:12.104765+0000","last_peered":"2023-12-12T09:52:12.104765+0000","last_clean":"2023-12-12T09:52:12.104765+0000","last_became_active":"2023-12-12T09:52:08.033745+0000","last_became_peered":"2023-12-12T09:52:08.033745+0000","last_unstale":"2023-12-12T09:52:12.104765+0000","last_undegraded":"2023-12-12T09:52:12.104765+0000","last_fullsized":"2023-12-12T09:52:12.104765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:51.434174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4637,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":98899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":98933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:52:37.814 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:52:38.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.065+0000 7f69bde7c640 1 Processor -- start 2023-12-12T09:52:38.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.065+0000 7f69bde7c640 1 -- start start 2023-12-12T09:52:38.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.067+0000 7f69bde7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69b8143e00 0x7f69b80b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.067+0000 7f69bde7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8072180 0x7f69b80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.067+0000 7f69bde7c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b80b3bb0 con 0x7f69b806c410 2023-12-12T09:52:38.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.067+0000 7f69bde7c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b81441e0 con 0x7f69b8143e00 2023-12-12T09:52:38.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.067+0000 7f69bde7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b8072680 con 0x7f69b8072180 2023-12-12T09:52:38.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.067+0000 7f69b7fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f69b806c410 0x7f69b8070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52352/0 (socket says 172.21.15.72:52352) 2023-12-12T09:52:38.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.067+0000 7f69b7fff640 1 -- 172.21.15.72:0/278929366 learned_addr learned my addr 172.21.15.72:0/278929366 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:38.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.067+0000 7f69b77fe640 1 --2- 172.21.15.72:0/278929366 >> v2:172.21.15.113:3300/0 conn(0x7f69b8143e00 0x7f69b80b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69aeffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8072180 0x7f69b80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452509172 0 0) 0x7f69b80b3bb0 con 0x7f69b806c410 2023-12-12T09:52:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6998003520 con 0x7f69b806c410 2023-12-12T09:52:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1574914383 0 0) 0x7f6998003520 con 0x7f69b806c410 2023-12-12T09:52:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8072180 msgr2=0x7f69b80b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8072180 0x7f69b80b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 >> v2:172.21.15.113:3300/0 conn(0x7f69b8143e00 msgr2=0x7f69b80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 --2- 172.21.15.72:0/278929366 >> v2:172.21.15.113:3300/0 conn(0x7f69b8143e00 0x7f69b80b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b80b1e30 con 0x7f69b806c410 2023-12-12T09:52:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f69b00019f0 con 0x7f69b806c410 2023-12-12T09:52:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 >> v1:172.21.15.72:6789/0 conn(0x7f69b806c410 legacy=0x7f69b8070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8072180 0x7f6998003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6998004d00 0x7f6998005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b81441e0 con 0x7f6998003ef0 2023-12-12T09:52:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b8072680 con 0x7f6998004d00 2023-12-12T09:52:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.068+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699800b980 con 0x7f69b8072180 2023-12-12T09:52:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.069+0000 7f69aeffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.069+0000 7f69b77fe640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8072180 0x7f6998003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.069+0000 7f69b7fff640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6998004d00 0x7f6998005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.069+0000 7f69aeffd640 1 -- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8072180 msgr2=0x7f6998003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.069+0000 7f69aeffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8072180 0x7f6998003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.069+0000 7f69aeffd640 1 -- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6998004d00 msgr2=0x7f6998005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.069+0000 7f69aeffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6998004d00 0x7f6998005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.069+0000 7f69aeffd640 1 -- 172.21.15.72:0/278929366 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6998003520 con 0x7f6998003ef0 2023-12-12T09:52:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.070+0000 7f69aeffd640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69980047c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009ac0 tx=0x7f69980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c226cb6b79522737 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.070+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a002c6b0 con 0x7f6998003ef0 2023-12-12T09:52:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.070+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a002c810 con 0x7f6998003ef0 2023-12-12T09:52:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.070+0000 7f69b6ffd640 1 -- 172.21.15.72:0/278929366 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a002cba0 con 0x7f6998003ef0 2023-12-12T09:52:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.071+0000 7f69bde7c640 1 -- 172.21.15.72:0/278929366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 msgr2=0x7f69980047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.071+0000 7f69bde7c640 1 --2- 172.21.15.72:0/278929366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69980047c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009ac0 tx=0x7f69980057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.071+0000 7f69b7fff640 1 -- 172.21.15.72:0/278929366 reap_dead start 2023-12-12T09:52:38.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.071+0000 7f69bde7c640 1 -- 172.21.15.72:0/278929366 shutdown_connections 2023-12-12T09:52:38.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.071+0000 7f69bde7c640 1 -- 172.21.15.72:0/278929366 >> 172.21.15.72:0/278929366 conn(0x7f69b80b0a90 msgr2=0x7f69b80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:38.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.071+0000 7f69bde7c640 1 -- 172.21.15.72:0/278929366 shutdown_connections 2023-12-12T09:52:38.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.071+0000 7f69bde7c640 1 -- 172.21.15.72:0/278929366 wait complete. 2023-12-12T09:52:38.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.073+0000 7f69bde7c640 1 Processor -- start 2023-12-12T09:52:38.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.073+0000 7f69bde7c640 1 -- start start 2023-12-12T09:52:38.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69bde7c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69b8142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69bde7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8140450 0x7f69b8142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69bde7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b815a750 0x7f69b815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69bde7c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b81f5900 con 0x7f6998003ef0 2023-12-12T09:52:38.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69bde7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b81f4b60 con 0x7f69b815a750 2023-12-12T09:52:38.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69bde7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b81408b0 con 0x7f69b8140450 2023-12-12T09:52:38.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69b8142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69b8142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40238/0 (socket says 172.21.15.72:40238) 2023-12-12T09:52:38.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b77fe640 1 -- 172.21.15.72:0/2825648909 learned_addr learned my addr 172.21.15.72:0/2825648909 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:38.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69aeffd640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8140450 0x7f69b8142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b7fff640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b815a750 0x7f69b815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b77fe640 1 -- 172.21.15.72:0/2825648909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8140450 msgr2=0x7f69b8142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b77fe640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8140450 0x7f69b8142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b77fe640 1 -- 172.21.15.72:0/2825648909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b815a750 msgr2=0x7f69b815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b77fe640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b815a750 0x7f69b815aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.074+0000 7f69b77fe640 1 -- 172.21.15.72:0/2825648909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b81f4e20 con 0x7f6998003ef0 2023-12-12T09:52:38.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.075+0000 7f69b77fe640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69b8142290 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69a80108e0 tx=0x7f69a8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:38.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.075+0000 7f69b4ff9640 1 -- 172.21.15.72:0/2825648909 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a800eb60 con 0x7f6998003ef0 2023-12-12T09:52:38.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.075+0000 7f69b4ff9640 1 -- 172.21.15.72:0/2825648909 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a800ecc0 con 0x7f6998003ef0 2023-12-12T09:52:38.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.075+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b81f5100 con 0x7f6998003ef0 2023-12-12T09:52:38.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.075+0000 7f69b4ff9640 1 -- 172.21.15.72:0/2825648909 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a801c400 con 0x7f6998003ef0 2023-12-12T09:52:38.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.075+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b815b0e0 con 0x7f6998003ef0 2023-12-12T09:52:38.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.076+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b80b2a10 con 0x7f6998003ef0 2023-12-12T09:52:38.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.077+0000 7f69b4ff9640 1 -- 172.21.15.72:0/2825648909 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69a801c560 con 0x7f6998003ef0 2023-12-12T09:52:38.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.078+0000 7f69b4ff9640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.078+0000 7f69b4ff9640 1 -- 172.21.15.72:0/2825648909 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69a808a080 con 0x7f6998003ef0 2023-12-12T09:52:38.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.078+0000 7f69aeffd640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.079+0000 7f69aeffd640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f699c06ae40 0x7f699c06d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69a0036000 tx=0x7f69a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:38.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.082+0000 7f69b4ff9640 1 -- 172.21.15.72:0/2825648909 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69a80bce90 con 0x7f6998003ef0 2023-12-12T09:52:38.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.254+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69b80b23f0 con 0x7f699c06ae40 2023-12-12T09:52:38.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.269+0000 7f69b4ff9640 1 -- 172.21.15.72:0/2825648909 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345154 (secure 0 0 0) 0x7f69a0034440 con 0x7f699c06ae40 2023-12-12T09:52:38.271 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:38.278 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:52:38.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.277+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f699c06ae40 msgr2=0x7f699c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.277+0000 7f69bde7c640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f699c06ae40 0x7f699c06d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69a0036000 tx=0x7f69a0030000 comp rx=0 tx=0).stop 2023-12-12T09:52:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 msgr2=0x7f69b8142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69b8142290 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69a80108e0 tx=0x7f69a8010da0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 shutdown_connections 2023-12-12T09:52:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b815a750 0x7f69b815aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b8140450 0x7f69b8142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 --2- 172.21.15.72:0/2825648909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6998003ef0 0x7f69b8142290 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 >> 172.21.15.72:0/2825648909 conn(0x7f69b80b0a90 msgr2=0x7f69b80b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:38.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 shutdown_connections 2023-12-12T09:52:38.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.278+0000 7f69bde7c640 1 -- 172.21.15.72:0/2825648909 wait complete. 2023-12-12T09:52:38.295 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-12-12T09:52:37.554883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3196,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3272704,"data_stored":1985289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004648"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070257+0000","last_change":"2023-12-12T09:52:07.005949+0000","last_active":"2023-12-12T09:52:12.070257+0000","last_peered":"2023-12-12T09:52:12.070257+0000","last_clean":"2023-12-12T09:52:12.070257+0000","last_became_active":"2023-12-12T09:52:07.005677+0000","last_became_peered":"2023-12-12T09:52:07.005677+0000","last_unstale":"2023-12-12T09:52:12.070257+0000","last_undegraded":"2023-12-12T09:52:12.070257+0000","last_fullsized":"2023-12-12T09:52:12.070257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:31.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.058906+0000","last_change":"2023-12-12T09:52:08.029768+0000","last_active":"2023-12-12T09:52:12.058906+0000","last_peered":"2023-12-12T09:52:12.058906+0000","last_clean":"2023-12-12T09:52:12.058906+0000","last_became_active":"2023-12-12T09:52:08.029493+0000","last_became_peered":"2023-12-12T09:52:08.029493+0000","last_unstale":"2023-12-12T09:52:12.058906+0000","last_undegraded":"2023-12-12T09:52:12.058906+0000","last_fullsized":"2023-12-12T09:52:12.058906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283436+0000","last_change":"2023-12-12T09:52:12.106967+0000","last_active":"2023-12-12T09:52:13.283436+0000","last_peered":"2023-12-12T09:52:13.283436+0000","last_clean":"2023-12-12T09:52:13.283436+0000","last_became_active":"2023-12-12T09:52:12.106583+0000","last_became_peered":"2023-12-12T09:52:12.106583+0000","last_unstale":"2023-12-12T09:52:13.283436+0000","last_undegraded":"2023-12-12T09:52:13.283436+0000","last_fullsized":"2023-12-12T09:52:13.283436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:02.585452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059029+0000","last_change":"2023-12-12T09:52:06.990826+0000","last_active":"2023-12-12T09:52:12.059029+0000","last_peered":"2023-12-12T09:52:12.059029+0000","last_clean":"2023-12-12T09:52:12.059029+0000","last_became_active":"2023-12-12T09:52:06.990515+0000","last_became_peered":"2023-12-12T09:52:06.990515+0000","last_unstale":"2023-12-12T09:52:12.059029+0000","last_undegraded":"2023-12-12T09:52:12.059029+0000","last_fullsized":"2023-12-12T09:52:12.059029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:19.282948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070081+0000","last_change":"2023-12-12T09:52:08.023719+0000","last_active":"2023-12-12T09:52:12.070081+0000","last_peered":"2023-12-12T09:52:12.070081+0000","last_clean":"2023-12-12T09:52:12.070081+0000","last_became_active":"2023-12-12T09:52:08.023372+0000","last_became_peered":"2023-12-12T09:52:08.023372+0000","last_unstale":"2023-12-12T09:52:12.070081+0000","last_undegraded":"2023-12-12T09:52:12.070081+0000","last_fullsized":"2023-12-12T09:52:12.070081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281424+0000","last_change":"2023-12-12T09:52:12.082594+0000","last_active":"2023-12-12T09:52:13.281424+0000","last_peered":"2023-12-12T09:52:13.281424+0000","last_clean":"2023-12-12T09:52:13.281424+0000","last_became_active":"2023-12-12T09:52:12.081951+0000","last_became_peered":"2023-12-12T09:52:12.081951+0000","last_unstale":"2023-12-12T09:52:13.281424+0000","last_undegraded":"2023-12-12T09:52:13.281424+0000","last_fullsized":"2023-12-12T09:52:13.281424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:45.079238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104327+0000","last_change":"2023-12-12T09:52:07.014029+0000","last_active":"2023-12-12T09:52:12.104327+0000","last_peered":"2023-12-12T09:52:12.104327+0000","last_clean":"2023-12-12T09:52:12.104327+0000","last_became_active":"2023-12-12T09:52:07.013726+0000","last_became_peered":"2023-12-12T09:52:07.013726+0000","last_unstale":"2023-12-12T09:52:12.104327+0000","last_undegraded":"2023-12-12T09:52:12.104327+0000","last_fullsized":"2023-12-12T09:52:12.104327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:46.546348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074859+0000","last_change":"2023-12-12T09:52:08.024974+0000","last_active":"2023-12-12T09:52:12.074859+0000","last_peered":"2023-12-12T09:52:12.074859+0000","last_clean":"2023-12-12T09:52:12.074859+0000","last_became_active":"2023-12-12T09:52:08.024739+0000","last_became_peered":"2023-12-12T09:52:08.024739+0000","last_unstale":"2023-12-12T09:52:12.074859+0000","last_undegraded":"2023-12-12T09:52:12.074859+0000","last_fullsized":"2023-12-12T09:52:12.074859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.950798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281819+0000","last_change":"2023-12-12T09:52:12.082902+0000","last_active":"2023-12-12T09:52:13.281819+0000","last_peered":"2023-12-12T09:52:13.281819+0000","last_clean":"2023-12-12T09:52:13.281819+0000","last_became_active":"2023-12-12T09:52:12.082411+0000","last_became_peered":"2023-12-12T09:52:12.082411+0000","last_unstale":"2023-12-12T09:52:13.281819+0000","last_undegraded":"2023-12-12T09:52:13.281819+0000","last_fullsized":"2023-12-12T09:52:13.281819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:39.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104053+0000","last_change":"2023-12-12T09:52:07.014406+0000","last_active":"2023-12-12T09:52:12.104053+0000","last_peered":"2023-12-12T09:52:12.104053+0000","last_clean":"2023-12-12T09:52:12.104053+0000","last_became_active":"2023-12-12T09:52:07.014067+0000","last_became_peered":"2023-12-12T09:52:07.014067+0000","last_unstale":"2023-12-12T09:52:12.104053+0000","last_undegraded":"2023-12-12T09:52:12.104053+0000","last_fullsized":"2023-12-12T09:52:12.104053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075313+0000","last_change":"2023-12-12T09:52:08.025152+0000","last_active":"2023-12-12T09:52:12.075313+0000","last_peered":"2023-12-12T09:52:12.075313+0000","last_clean":"2023-12-12T09:52:12.075313+0000","last_became_active":"2023-12-12T09:52:08.024282+0000","last_became_peered":"2023-12-12T09:52:08.024282+0000","last_unstale":"2023-12-12T09:52:12.075313+0000","last_undegraded":"2023-12-12T09:52:12.075313+0000","last_fullsized":"2023-12-12T09:52:12.075313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:35.813548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281955+0000","last_change":"2023-12-12T09:52:12.079381+0000","last_active":"2023-12-12T09:52:13.281955+0000","last_peered":"2023-12-12T09:52:13.281955+0000","last_clean":"2023-12-12T09:52:13.281955+0000","last_became_active":"2023-12-12T09:52:12.078805+0000","last_became_peered":"2023-12-12T09:52:12.078805+0000","last_unstale":"2023-12-12T09:52:13.281955+0000","last_undegraded":"2023-12-12T09:52:13.281955+0000","last_fullsized":"2023-12-12T09:52:13.281955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:21:02.014413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050630+0000","last_change":"2023-12-12T09:52:08.020343+0000","last_active":"2023-12-12T09:52:12.050630+0000","last_peered":"2023-12-12T09:52:12.050630+0000","last_clean":"2023-12-12T09:52:12.050630+0000","last_became_active":"2023-12-12T09:52:08.019981+0000","last_became_peered":"2023-12-12T09:52:08.019981+0000","last_unstale":"2023-12-12T09:52:12.050630+0000","last_undegraded":"2023-12-12T09:52:12.050630+0000","last_fullsized":"2023-12-12T09:52:12.050630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104677+0000","last_change":"2023-12-12T09:52:07.011835+0000","last_active":"2023-12-12T09:52:12.104677+0000","last_peered":"2023-12-12T09:52:12.104677+0000","last_clean":"2023-12-12T09:52:12.104677+0000","last_became_active":"2023-12-12T09:52:07.011505+0000","last_became_peered":"2023-12-12T09:52:07.011505+0000","last_unstale":"2023-12-12T09:52:12.104677+0000","last_undegraded":"2023-12-12T09:52:12.104677+0000","last_fullsized":"2023-12-12T09:52:12.104677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285005+0000","last_change":"2023-12-12T09:52:12.080462+0000","last_active":"2023-12-12T09:52:13.285005+0000","last_peered":"2023-12-12T09:52:13.285005+0000","last_clean":"2023-12-12T09:52:13.285005+0000","last_became_active":"2023-12-12T09:52:12.080128+0000","last_became_peered":"2023-12-12T09:52:12.080128+0000","last_unstale":"2023-12-12T09:52:13.285005+0000","last_undegraded":"2023-12-12T09:52:13.285005+0000","last_fullsized":"2023-12-12T09:52:13.285005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054727+0000","last_change":"2023-12-12T09:52:08.028847+0000","last_active":"2023-12-12T09:52:12.054727+0000","last_peered":"2023-12-12T09:52:12.054727+0000","last_clean":"2023-12-12T09:52:12.054727+0000","last_became_active":"2023-12-12T09:52:08.028537+0000","last_became_peered":"2023-12-12T09:52:08.028537+0000","last_unstale":"2023-12-12T09:52:12.054727+0000","last_undegraded":"2023-12-12T09:52:12.054727+0000","last_fullsized":"2023-12-12T09:52:12.054727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068568+0000","last_change":"2023-12-12T09:52:06.992502+0000","last_active":"2023-12-12T09:52:12.068568+0000","last_peered":"2023-12-12T09:52:12.068568+0000","last_clean":"2023-12-12T09:52:12.068568+0000","last_became_active":"2023-12-12T09:52:06.992218+0000","last_became_peered":"2023-12-12T09:52:06.992218+0000","last_unstale":"2023-12-12T09:52:12.068568+0000","last_undegraded":"2023-12-12T09:52:12.068568+0000","last_fullsized":"2023-12-12T09:52:12.068568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284506+0000","last_change":"2023-12-12T09:52:12.083926+0000","last_active":"2023-12-12T09:52:13.284506+0000","last_peered":"2023-12-12T09:52:13.284506+0000","last_clean":"2023-12-12T09:52:13.284506+0000","last_became_active":"2023-12-12T09:52:12.083322+0000","last_became_peered":"2023-12-12T09:52:12.083322+0000","last_unstale":"2023-12-12T09:52:13.284506+0000","last_undegraded":"2023-12-12T09:52:13.284506+0000","last_fullsized":"2023-12-12T09:52:13.284506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:31.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059740+0000","last_change":"2023-12-12T09:52:08.021868+0000","last_active":"2023-12-12T09:52:12.059740+0000","last_peered":"2023-12-12T09:52:12.059740+0000","last_clean":"2023-12-12T09:52:12.059740+0000","last_became_active":"2023-12-12T09:52:08.020477+0000","last_became_peered":"2023-12-12T09:52:08.020477+0000","last_unstale":"2023-12-12T09:52:12.059740+0000","last_undegraded":"2023-12-12T09:52:12.059740+0000","last_fullsized":"2023-12-12T09:52:12.059740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055893+0000","last_change":"2023-12-12T09:52:07.005983+0000","last_active":"2023-12-12T09:52:12.055893+0000","last_peered":"2023-12-12T09:52:12.055893+0000","last_clean":"2023-12-12T09:52:12.055893+0000","last_became_active":"2023-12-12T09:52:07.005723+0000","last_became_peered":"2023-12-12T09:52:07.005723+0000","last_unstale":"2023-12-12T09:52:12.055893+0000","last_undegraded":"2023-12-12T09:52:12.055893+0000","last_fullsized":"2023-12-12T09:52:12.055893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282866+0000","last_change":"2023-12-12T09:52:12.110411+0000","last_active":"2023-12-12T09:52:13.282866+0000","last_peered":"2023-12-12T09:52:13.282866+0000","last_clean":"2023-12-12T09:52:13.282866+0000","last_became_active":"2023-12-12T09:52:12.110045+0000","last_became_peered":"2023-12-12T09:52:12.110045+0000","last_unstale":"2023-12-12T09:52:13.282866+0000","last_undegraded":"2023-12-12T09:52:13.282866+0000","last_fullsized":"2023-12-12T09:52:13.282866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:39.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.069034+0000","last_change":"2023-12-12T09:52:08.034974+0000","last_active":"2023-12-12T09:52:12.069034+0000","last_peered":"2023-12-12T09:52:12.069034+0000","last_clean":"2023-12-12T09:52:12.069034+0000","last_became_active":"2023-12-12T09:52:08.034169+0000","last_became_peered":"2023-12-12T09:52:08.034169+0000","last_unstale":"2023-12-12T09:52:12.069034+0000","last_undegraded":"2023-12-12T09:52:12.069034+0000","last_fullsized":"2023-12-12T09:52:12.069034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:47:41.840110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103698+0000","last_change":"2023-12-12T09:52:07.006046+0000","last_active":"2023-12-12T09:52:12.103698+0000","last_peered":"2023-12-12T09:52:12.103698+0000","last_clean":"2023-12-12T09:52:12.103698+0000","last_became_active":"2023-12-12T09:52:07.005742+0000","last_became_peered":"2023-12-12T09:52:07.005742+0000","last_unstale":"2023-12-12T09:52:12.103698+0000","last_undegraded":"2023-12-12T09:52:12.103698+0000","last_fullsized":"2023-12-12T09:52:12.103698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:31.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285141+0000","last_change":"2023-12-12T09:52:12.084173+0000","last_active":"2023-12-12T09:52:13.285141+0000","last_peered":"2023-12-12T09:52:13.285141+0000","last_clean":"2023-12-12T09:52:13.285141+0000","last_became_active":"2023-12-12T09:52:12.083324+0000","last_became_peered":"2023-12-12T09:52:12.083324+0000","last_unstale":"2023-12-12T09:52:13.285141+0000","last_undegraded":"2023-12-12T09:52:13.285141+0000","last_fullsized":"2023-12-12T09:52:13.285141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.071231+0000","last_change":"2023-12-12T09:52:08.022538+0000","last_active":"2023-12-12T09:52:12.071231+0000","last_peered":"2023-12-12T09:52:12.071231+0000","last_clean":"2023-12-12T09:52:12.071231+0000","last_became_active":"2023-12-12T09:52:08.022274+0000","last_became_peered":"2023-12-12T09:52:08.022274+0000","last_unstale":"2023-12-12T09:52:12.071231+0000","last_undegraded":"2023-12-12T09:52:12.071231+0000","last_fullsized":"2023-12-12T09:52:12.071231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053881+0000","last_change":"2023-12-12T09:52:07.012094+0000","last_active":"2023-12-12T09:52:12.053881+0000","last_peered":"2023-12-12T09:52:12.053881+0000","last_clean":"2023-12-12T09:52:12.053881+0000","last_became_active":"2023-12-12T09:52:07.011693+0000","last_became_peered":"2023-12-12T09:52:07.011693+0000","last_unstale":"2023-12-12T09:52:12.053881+0000","last_undegraded":"2023-12-12T09:52:12.053881+0000","last_fullsized":"2023-12-12T09:52:12.053881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:47.890555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281952+0000","last_change":"2023-12-12T09:52:12.111616+0000","last_active":"2023-12-12T09:52:13.281952+0000","last_peered":"2023-12-12T09:52:13.281952+0000","last_clean":"2023-12-12T09:52:13.281952+0000","last_became_active":"2023-12-12T09:52:12.110326+0000","last_became_peered":"2023-12-12T09:52:12.110326+0000","last_unstale":"2023-12-12T09:52:13.281952+0000","last_undegraded":"2023-12-12T09:52:13.281952+0000","last_fullsized":"2023-12-12T09:52:13.281952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:34:01.832456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.063408+0000","last_change":"2023-12-12T09:52:08.034520+0000","last_active":"2023-12-12T09:52:12.063408+0000","last_peered":"2023-12-12T09:52:12.063408+0000","last_clean":"2023-12-12T09:52:12.063408+0000","last_became_active":"2023-12-12T09:52:08.034232+0000","last_became_peered":"2023-12-12T09:52:08.034232+0000","last_unstale":"2023-12-12T09:52:12.063408+0000","last_undegraded":"2023-12-12T09:52:12.063408+0000","last_fullsized":"2023-12-12T09:52:12.063408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.760548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.057080+0000","last_change":"2023-12-12T09:52:07.004065+0000","last_active":"2023-12-12T09:52:12.057080+0000","last_peered":"2023-12-12T09:52:12.057080+0000","last_clean":"2023-12-12T09:52:12.057080+0000","last_became_active":"2023-12-12T09:52:07.003690+0000","last_became_peered":"2023-12-12T09:52:07.003690+0000","last_unstale":"2023-12-12T09:52:12.057080+0000","last_undegraded":"2023-12-12T09:52:12.057080+0000","last_fullsized":"2023-12-12T09:52:12.057080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:47.890555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285159+0000","last_change":"2023-12-12T09:52:12.108409+0000","last_active":"2023-12-12T09:52:13.285159+0000","last_peered":"2023-12-12T09:52:13.285159+0000","last_clean":"2023-12-12T09:52:13.285159+0000","last_became_active":"2023-12-12T09:52:12.107934+0000","last_became_peered":"2023-12-12T09:52:12.107934+0000","last_unstale":"2023-12-12T09:52:13.285159+0000","last_undegraded":"2023-12-12T09:52:13.285159+0000","last_fullsized":"2023-12-12T09:52:13.285159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:20.552141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065611+0000","last_change":"2023-12-12T09:52:08.031660+0000","last_active":"2023-12-12T09:52:12.065611+0000","last_peered":"2023-12-12T09:52:12.065611+0000","last_clean":"2023-12-12T09:52:12.065611+0000","last_became_active":"2023-12-12T09:52:08.030341+0000","last_became_peered":"2023-12-12T09:52:08.030341+0000","last_unstale":"2023-12-12T09:52:12.065611+0000","last_undegraded":"2023-12-12T09:52:12.065611+0000","last_fullsized":"2023-12-12T09:52:12.065611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104424+0000","last_change":"2023-12-12T09:52:07.014640+0000","last_active":"2023-12-12T09:52:12.104424+0000","last_peered":"2023-12-12T09:52:12.104424+0000","last_clean":"2023-12-12T09:52:12.104424+0000","last_became_active":"2023-12-12T09:52:07.014313+0000","last_became_peered":"2023-12-12T09:52:07.014313+0000","last_unstale":"2023-12-12T09:52:12.104424+0000","last_undegraded":"2023-12-12T09:52:12.104424+0000","last_fullsized":"2023-12-12T09:52:12.104424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:25.514763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285307+0000","last_change":"2023-12-12T09:52:12.111140+0000","last_active":"2023-12-12T09:52:13.285307+0000","last_peered":"2023-12-12T09:52:13.285307+0000","last_clean":"2023-12-12T09:52:13.285307+0000","last_became_active":"2023-12-12T09:52:12.110437+0000","last_became_peered":"2023-12-12T09:52:12.110437+0000","last_unstale":"2023-12-12T09:52:13.285307+0000","last_undegraded":"2023-12-12T09:52:13.285307+0000","last_fullsized":"2023-12-12T09:52:13.285307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075331+0000","last_change":"2023-12-12T09:52:08.031638+0000","last_active":"2023-12-12T09:52:12.075331+0000","last_peered":"2023-12-12T09:52:12.075331+0000","last_clean":"2023-12-12T09:52:12.075331+0000","last_became_active":"2023-12-12T09:52:08.031330+0000","last_became_peered":"2023-12-12T09:52:08.031330+0000","last_unstale":"2023-12-12T09:52:12.075331+0000","last_undegraded":"2023-12-12T09:52:12.075331+0000","last_fullsized":"2023-12-12T09:52:12.075331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:27.371886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065722+0000","last_change":"2023-12-12T09:52:06.996775+0000","last_active":"2023-12-12T09:52:12.065722+0000","last_peered":"2023-12-12T09:52:12.065722+0000","last_clean":"2023-12-12T09:52:12.065722+0000","last_became_active":"2023-12-12T09:52:06.995618+0000","last_became_peered":"2023-12-12T09:52:06.995618+0000","last_unstale":"2023-12-12T09:52:12.065722+0000","last_undegraded":"2023-12-12T09:52:12.065722+0000","last_fullsized":"2023-12-12T09:52:12.065722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:26:34.669025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282930+0000","last_change":"2023-12-12T09:52:12.085685+0000","last_active":"2023-12-12T09:52:13.282930+0000","last_peered":"2023-12-12T09:52:13.282930+0000","last_clean":"2023-12-12T09:52:13.282930+0000","last_became_active":"2023-12-12T09:52:12.085247+0000","last_became_peered":"2023-12-12T09:52:12.085247+0000","last_unstale":"2023-12-12T09:52:13.282930+0000","last_undegraded":"2023-12-12T09:52:13.282930+0000","last_fullsized":"2023-12-12T09:52:13.282930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:31.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070066+0000","last_change":"2023-12-12T09:52:08.020254+0000","last_active":"2023-12-12T09:52:12.070066+0000","last_peered":"2023-12-12T09:52:12.070066+0000","last_clean":"2023-12-12T09:52:12.070066+0000","last_became_active":"2023-12-12T09:52:08.019979+0000","last_became_peered":"2023-12-12T09:52:08.019979+0000","last_unstale":"2023-12-12T09:52:12.070066+0000","last_undegraded":"2023-12-12T09:52:12.070066+0000","last_fullsized":"2023-12-12T09:52:12.070066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:32.163069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074498+0000","last_change":"2023-12-12T09:52:06.997560+0000","last_active":"2023-12-12T09:52:12.074498+0000","last_peered":"2023-12-12T09:52:12.074498+0000","last_clean":"2023-12-12T09:52:12.074498+0000","last_became_active":"2023-12-12T09:52:06.996940+0000","last_became_peered":"2023-12-12T09:52:06.996940+0000","last_unstale":"2023-12-12T09:52:12.074498+0000","last_undegraded":"2023-12-12T09:52:12.074498+0000","last_fullsized":"2023-12-12T09:52:12.074498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:09.895493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285696+0000","last_change":"2023-12-12T09:52:12.077447+0000","last_active":"2023-12-12T09:52:13.285696+0000","last_peered":"2023-12-12T09:52:13.285696+0000","last_clean":"2023-12-12T09:52:13.285696+0000","last_became_active":"2023-12-12T09:52:12.076927+0000","last_became_peered":"2023-12-12T09:52:12.076927+0000","last_unstale":"2023-12-12T09:52:13.285696+0000","last_undegraded":"2023-12-12T09:52:13.285696+0000","last_fullsized":"2023-12-12T09:52:13.285696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:55.497789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.057627+0000","last_change":"2023-12-12T09:52:08.035111+0000","last_active":"2023-12-12T09:52:12.057627+0000","last_peered":"2023-12-12T09:52:12.057627+0000","last_clean":"2023-12-12T09:52:12.057627+0000","last_became_active":"2023-12-12T09:52:08.034823+0000","last_became_peered":"2023-12-12T09:52:08.034823+0000","last_unstale":"2023-12-12T09:52:12.057627+0000","last_undegraded":"2023-12-12T09:52:12.057627+0000","last_fullsized":"2023-12-12T09:52:12.057627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055046+0000","last_change":"2023-12-12T09:52:06.993403+0000","last_active":"2023-12-12T09:52:12.055046+0000","last_peered":"2023-12-12T09:52:12.055046+0000","last_clean":"2023-12-12T09:52:12.055046+0000","last_became_active":"2023-12-12T09:52:06.993116+0000","last_became_peered":"2023-12-12T09:52:06.993116+0000","last_unstale":"2023-12-12T09:52:12.055046+0000","last_undegraded":"2023-12-12T09:52:12.055046+0000","last_fullsized":"2023-12-12T09:52:12.055046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283254+0000","last_change":"2023-12-12T09:52:12.080184+0000","last_active":"2023-12-12T09:52:13.283254+0000","last_peered":"2023-12-12T09:52:13.283254+0000","last_clean":"2023-12-12T09:52:13.283254+0000","last_became_active":"2023-12-12T09:52:12.079804+0000","last_became_peered":"2023-12-12T09:52:12.079804+0000","last_unstale":"2023-12-12T09:52:13.283254+0000","last_undegraded":"2023-12-12T09:52:13.283254+0000","last_fullsized":"2023-12-12T09:52:13.283254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065376+0000","last_change":"2023-12-12T09:52:08.026908+0000","last_active":"2023-12-12T09:52:12.065376+0000","last_peered":"2023-12-12T09:52:12.065376+0000","last_clean":"2023-12-12T09:52:12.065376+0000","last_became_active":"2023-12-12T09:52:08.025912+0000","last_became_peered":"2023-12-12T09:52:08.025912+0000","last_unstale":"2023-12-12T09:52:12.065376+0000","last_undegraded":"2023-12-12T09:52:12.065376+0000","last_fullsized":"2023-12-12T09:52:12.065376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103688+0000","last_change":"2023-12-12T09:52:07.009290+0000","last_active":"2023-12-12T09:52:12.103688+0000","last_peered":"2023-12-12T09:52:12.103688+0000","last_clean":"2023-12-12T09:52:12.103688+0000","last_became_active":"2023-12-12T09:52:07.008896+0000","last_became_peered":"2023-12-12T09:52:07.008896+0000","last_unstale":"2023-12-12T09:52:12.103688+0000","last_undegraded":"2023-12-12T09:52:12.103688+0000","last_fullsized":"2023-12-12T09:52:12.103688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:21.191123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.290045+0000","last_change":"2023-12-12T09:52:12.115265+0000","last_active":"2023-12-12T09:52:13.290045+0000","last_peered":"2023-12-12T09:52:13.290045+0000","last_clean":"2023-12-12T09:52:13.290045+0000","last_became_active":"2023-12-12T09:52:12.114674+0000","last_became_peered":"2023-12-12T09:52:12.114674+0000","last_unstale":"2023-12-12T09:52:13.290045+0000","last_undegraded":"2023-12-12T09:52:13.290045+0000","last_fullsized":"2023-12-12T09:52:13.290045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:34:54.772153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060809+0000","last_change":"2023-12-12T09:52:08.028299+0000","last_active":"2023-12-12T09:52:12.060809+0000","last_peered":"2023-12-12T09:52:12.060809+0000","last_clean":"2023-12-12T09:52:12.060809+0000","last_became_active":"2023-12-12T09:52:08.028080+0000","last_became_peered":"2023-12-12T09:52:08.028080+0000","last_unstale":"2023-12-12T09:52:12.060809+0000","last_undegraded":"2023-12-12T09:52:12.060809+0000","last_fullsized":"2023-12-12T09:52:12.060809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074571+0000","last_change":"2023-12-12T09:52:06.996346+0000","last_active":"2023-12-12T09:52:12.074571+0000","last_peered":"2023-12-12T09:52:12.074571+0000","last_clean":"2023-12-12T09:52:12.074571+0000","last_became_active":"2023-12-12T09:52:06.995462+0000","last_became_peered":"2023-12-12T09:52:06.995462+0000","last_unstale":"2023-12-12T09:52:12.074571+0000","last_undegraded":"2023-12-12T09:52:12.074571+0000","last_fullsized":"2023-12-12T09:52:12.074571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:59.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103796+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:52:12.103796+0000","last_peered":"2023-12-12T09:52:12.103796+0000","last_clean":"2023-12-12T09:52:12.103796+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:52:12.103796+0000","last_undegraded":"2023-12-12T09:52:12.103796+0000","last_fullsized":"2023-12-12T09:52:12.103796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.055422+0000","last_change":"2023-12-12T09:52:12.109683+0000","last_active":"2023-12-12T09:52:13.055422+0000","last_peered":"2023-12-12T09:52:13.055422+0000","last_clean":"2023-12-12T09:52:13.055422+0000","last_became_active":"2023-12-12T09:52:12.108319+0000","last_became_peered":"2023-12-12T09:52:12.108319+0000","last_unstale":"2023-12-12T09:52:13.055422+0000","last_undegraded":"2023-12-12T09:52:13.055422+0000","last_fullsized":"2023-12-12T09:52:13.055422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:34:01.832456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104240+0000","last_change":"2023-12-12T09:52:08.033828+0000","last_active":"2023-12-12T09:52:12.104240+0000","last_peered":"2023-12-12T09:52:12.104240+0000","last_clean":"2023-12-12T09:52:12.104240+0000","last_became_active":"2023-12-12T09:52:08.030396+0000","last_became_peered":"2023-12-12T09:52:08.030396+0000","last_unstale":"2023-12-12T09:52:12.104240+0000","last_undegraded":"2023-12-12T09:52:12.104240+0000","last_fullsized":"2023-12-12T09:52:12.104240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:27.371886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060933+0000","last_change":"2023-12-12T09:52:06.998515+0000","last_active":"2023-12-12T09:52:12.060933+0000","last_peered":"2023-12-12T09:52:12.060933+0000","last_clean":"2023-12-12T09:52:12.060933+0000","last_became_active":"2023-12-12T09:52:06.998220+0000","last_became_peered":"2023-12-12T09:52:06.998220+0000","last_unstale":"2023-12-12T09:52:12.060933+0000","last_undegraded":"2023-12-12T09:52:12.060933+0000","last_fullsized":"2023-12-12T09:52:12.060933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:19.282948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.054359+0000","last_change":"2023-12-12T09:52:12.106813+0000","last_active":"2023-12-12T09:52:13.054359+0000","last_peered":"2023-12-12T09:52:13.054359+0000","last_clean":"2023-12-12T09:52:13.054359+0000","last_became_active":"2023-12-12T09:52:12.106334+0000","last_became_peered":"2023-12-12T09:52:12.106334+0000","last_unstale":"2023-12-12T09:52:13.054359+0000","last_undegraded":"2023-12-12T09:52:13.054359+0000","last_fullsized":"2023-12-12T09:52:13.054359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:39.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074947+0000","last_change":"2023-12-12T09:52:08.033424+0000","last_active":"2023-12-12T09:52:12.074947+0000","last_peered":"2023-12-12T09:52:12.074947+0000","last_clean":"2023-12-12T09:52:12.074947+0000","last_became_active":"2023-12-12T09:52:08.030055+0000","last_became_peered":"2023-12-12T09:52:08.030055+0000","last_unstale":"2023-12-12T09:52:12.074947+0000","last_undegraded":"2023-12-12T09:52:12.074947+0000","last_fullsized":"2023-12-12T09:52:12.074947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:32.450565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056464+0000","last_change":"2023-12-12T09:52:07.000843+0000","last_active":"2023-12-12T09:52:12.056464+0000","last_peered":"2023-12-12T09:52:12.056464+0000","last_clean":"2023-12-12T09:52:12.056464+0000","last_became_active":"2023-12-12T09:52:07.000476+0000","last_became_peered":"2023-12-12T09:52:07.000476+0000","last_unstale":"2023-12-12T09:52:12.056464+0000","last_undegraded":"2023-12-12T09:52:12.056464+0000","last_fullsized":"2023-12-12T09:52:12.056464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:59.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285509+0000","last_change":"2023-12-12T09:52:12.082760+0000","last_active":"2023-12-12T09:52:13.285509+0000","last_peered":"2023-12-12T09:52:13.285509+0000","last_clean":"2023-12-12T09:52:13.285509+0000","last_became_active":"2023-12-12T09:52:12.082269+0000","last_became_peered":"2023-12-12T09:52:12.082269+0000","last_unstale":"2023-12-12T09:52:13.285509+0000","last_undegraded":"2023-12-12T09:52:13.285509+0000","last_fullsized":"2023-12-12T09:52:13.285509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104860+0000","last_change":"2023-12-12T09:52:08.035636+0000","last_active":"2023-12-12T09:52:12.104860+0000","last_peered":"2023-12-12T09:52:12.104860+0000","last_clean":"2023-12-12T09:52:12.104860+0000","last_became_active":"2023-12-12T09:52:08.035306+0000","last_became_peered":"2023-12-12T09:52:08.035306+0000","last_unstale":"2023-12-12T09:52:12.104860+0000","last_undegraded":"2023-12-12T09:52:12.104860+0000","last_fullsized":"2023-12-12T09:52:12.104860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:34:50.708538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053859+0000","last_change":"2023-12-12T09:52:06.993300+0000","last_active":"2023-12-12T09:52:12.053859+0000","last_peered":"2023-12-12T09:52:12.053859+0000","last_clean":"2023-12-12T09:52:12.053859+0000","last_became_active":"2023-12-12T09:52:06.992970+0000","last_became_peered":"2023-12-12T09:52:06.992970+0000","last_unstale":"2023-12-12T09:52:12.053859+0000","last_undegraded":"2023-12-12T09:52:12.053859+0000","last_fullsized":"2023-12-12T09:52:12.053859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:21.191123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.057884+0000","last_change":"2023-12-12T09:52:12.107302+0000","last_active":"2023-12-12T09:52:13.057884+0000","last_peered":"2023-12-12T09:52:13.057884+0000","last_clean":"2023-12-12T09:52:13.057884+0000","last_became_active":"2023-12-12T09:52:12.106810+0000","last_became_peered":"2023-12-12T09:52:12.106810+0000","last_unstale":"2023-12-12T09:52:13.057884+0000","last_undegraded":"2023-12-12T09:52:13.057884+0000","last_fullsized":"2023-12-12T09:52:13.057884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:25:03.440488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104689+0000","last_change":"2023-12-12T09:52:08.039951+0000","last_active":"2023-12-12T09:52:12.104689+0000","last_peered":"2023-12-12T09:52:12.104689+0000","last_clean":"2023-12-12T09:52:12.104689+0000","last_became_active":"2023-12-12T09:52:08.039124+0000","last_became_peered":"2023-12-12T09:52:08.039124+0000","last_unstale":"2023-12-12T09:52:12.104689+0000","last_undegraded":"2023-12-12T09:52:12.104689+0000","last_fullsized":"2023-12-12T09:52:12.104689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:32.450565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066373+0000","last_change":"2023-12-12T09:52:06.998901+0000","last_active":"2023-12-12T09:52:12.066373+0000","last_peered":"2023-12-12T09:52:12.066373+0000","last_clean":"2023-12-12T09:52:12.066373+0000","last_became_active":"2023-12-12T09:52:06.998458+0000","last_became_peered":"2023-12-12T09:52:06.998458+0000","last_unstale":"2023-12-12T09:52:12.066373+0000","last_undegraded":"2023-12-12T09:52:12.066373+0000","last_fullsized":"2023-12-12T09:52:12.066373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281456+0000","last_change":"2023-12-12T09:52:12.080392+0000","last_active":"2023-12-12T09:52:13.281456+0000","last_peered":"2023-12-12T09:52:13.281456+0000","last_clean":"2023-12-12T09:52:13.281456+0000","last_became_active":"2023-12-12T09:52:12.079941+0000","last_became_peered":"2023-12-12T09:52:12.079941+0000","last_unstale":"2023-12-12T09:52:13.281456+0000","last_undegraded":"2023-12-12T09:52:13.281456+0000","last_fullsized":"2023-12-12T09:52:13.281456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:10.824163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103824+0000","last_change":"2023-12-12T09:52:08.032605+0000","last_active":"2023-12-12T09:52:12.103824+0000","last_peered":"2023-12-12T09:52:12.103824+0000","last_clean":"2023-12-12T09:52:12.103824+0000","last_became_active":"2023-12-12T09:52:08.032321+0000","last_became_peered":"2023-12-12T09:52:08.032321+0000","last_unstale":"2023-12-12T09:52:12.103824+0000","last_undegraded":"2023-12-12T09:52:12.103824+0000","last_fullsized":"2023-12-12T09:52:12.103824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053766+0000","last_change":"2023-12-12T09:52:06.993710+0000","last_active":"2023-12-12T09:52:12.053766+0000","last_peered":"2023-12-12T09:52:12.053766+0000","last_clean":"2023-12-12T09:52:12.053766+0000","last_became_active":"2023-12-12T09:52:06.993427+0000","last_became_peered":"2023-12-12T09:52:06.993427+0000","last_unstale":"2023-12-12T09:52:12.053766+0000","last_undegraded":"2023-12-12T09:52:12.053766+0000","last_fullsized":"2023-12-12T09:52:12.053766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:39.005895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.289994+0000","last_change":"2023-12-12T09:52:12.083094+0000","last_active":"2023-12-12T09:52:13.289994+0000","last_peered":"2023-12-12T09:52:13.289994+0000","last_clean":"2023-12-12T09:52:13.289994+0000","last_became_active":"2023-12-12T09:52:12.082050+0000","last_became_peered":"2023-12-12T09:52:12.082050+0000","last_unstale":"2023-12-12T09:52:13.289994+0000","last_undegraded":"2023-12-12T09:52:13.289994+0000","last_fullsized":"2023-12-12T09:52:13.289994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:51.971159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050599+0000","last_change":"2023-12-12T09:52:08.020100+0000","last_active":"2023-12-12T09:52:12.050599+0000","last_peered":"2023-12-12T09:52:12.050599+0000","last_clean":"2023-12-12T09:52:12.050599+0000","last_became_active":"2023-12-12T09:52:08.019776+0000","last_became_peered":"2023-12-12T09:52:08.019776+0000","last_unstale":"2023-12-12T09:52:12.050599+0000","last_undegraded":"2023-12-12T09:52:12.050599+0000","last_fullsized":"2023-12-12T09:52:12.050599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050514+0000","last_change":"2023-12-12T09:52:07.002330+0000","last_active":"2023-12-12T09:52:12.050514+0000","last_peered":"2023-12-12T09:52:12.050514+0000","last_clean":"2023-12-12T09:52:12.050514+0000","last_became_active":"2023-12-12T09:52:07.001838+0000","last_became_peered":"2023-12-12T09:52:07.001838+0000","last_unstale":"2023-12-12T09:52:12.050514+0000","last_undegraded":"2023-12-12T09:52:12.050514+0000","last_fullsized":"2023-12-12T09:52:12.050514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:44.065611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.287475+0000","last_change":"2023-12-12T09:52:12.111595+0000","last_active":"2023-12-12T09:52:13.287475+0000","last_peered":"2023-12-12T09:52:13.287475+0000","last_clean":"2023-12-12T09:52:13.287475+0000","last_became_active":"2023-12-12T09:52:12.111143+0000","last_became_peered":"2023-12-12T09:52:12.111143+0000","last_unstale":"2023-12-12T09:52:13.287475+0000","last_undegraded":"2023-12-12T09:52:13.287475+0000","last_fullsized":"2023-12-12T09:52:13.287475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:07:22.383529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104101+0000","last_change":"2023-12-12T09:52:08.036655+0000","last_active":"2023-12-12T09:52:12.104101+0000","last_peered":"2023-12-12T09:52:12.104101+0000","last_clean":"2023-12-12T09:52:12.104101+0000","last_became_active":"2023-12-12T09:52:08.036260+0000","last_became_peered":"2023-12-12T09:52:08.036260+0000","last_unstale":"2023-12-12T09:52:12.104101+0000","last_undegraded":"2023-12-12T09:52:12.104101+0000","last_fullsized":"2023-12-12T09:52:12.104101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:35.813548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074437+0000","last_change":"2023-12-12T09:52:06.999233+0000","last_active":"2023-12-12T09:52:12.074437+0000","last_peered":"2023-12-12T09:52:12.074437+0000","last_clean":"2023-12-12T09:52:12.074437+0000","last_became_active":"2023-12-12T09:52:06.998962+0000","last_became_peered":"2023-12-12T09:52:06.998962+0000","last_unstale":"2023-12-12T09:52:12.074437+0000","last_undegraded":"2023-12-12T09:52:12.074437+0000","last_fullsized":"2023-12-12T09:52:12.074437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:25.514763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285663+0000","last_change":"2023-12-12T09:52:12.106941+0000","last_active":"2023-12-12T09:52:13.285663+0000","last_peered":"2023-12-12T09:52:13.285663+0000","last_clean":"2023-12-12T09:52:13.285663+0000","last_became_active":"2023-12-12T09:52:12.106547+0000","last_became_peered":"2023-12-12T09:52:12.106547+0000","last_unstale":"2023-12-12T09:52:13.285663+0000","last_undegraded":"2023-12-12T09:52:13.285663+0000","last_fullsized":"2023-12-12T09:52:13.285663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:04:27.791759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070910+0000","last_change":"2023-12-12T09:52:08.029830+0000","last_active":"2023-12-12T09:52:12.070910+0000","last_peered":"2023-12-12T09:52:12.070910+0000","last_clean":"2023-12-12T09:52:12.070910+0000","last_became_active":"2023-12-12T09:52:08.029455+0000","last_became_peered":"2023-12-12T09:52:08.029455+0000","last_unstale":"2023-12-12T09:52:12.070910+0000","last_undegraded":"2023-12-12T09:52:12.070910+0000","last_fullsized":"2023-12-12T09:52:12.070910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068345+0000","last_change":"2023-12-12T09:52:07.000933+0000","last_active":"2023-12-12T09:52:12.068345+0000","last_peered":"2023-12-12T09:52:12.068345+0000","last_clean":"2023-12-12T09:52:12.068345+0000","last_became_active":"2023-12-12T09:52:07.000646+0000","last_became_peered":"2023-12-12T09:52:07.000646+0000","last_unstale":"2023-12-12T09:52:12.068345+0000","last_undegraded":"2023-12-12T09:52:12.068345+0000","last_fullsized":"2023-12-12T09:52:12.068345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:44.065611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.059459+0000","last_change":"2023-12-12T09:52:12.110657+0000","last_active":"2023-12-12T09:52:13.059459+0000","last_peered":"2023-12-12T09:52:13.059459+0000","last_clean":"2023-12-12T09:52:13.059459+0000","last_became_active":"2023-12-12T09:52:12.109951+0000","last_became_peered":"2023-12-12T09:52:12.109951+0000","last_unstale":"2023-12-12T09:52:13.059459+0000","last_undegraded":"2023-12-12T09:52:13.059459+0000","last_fullsized":"2023-12-12T09:52:13.059459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:20.552141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104135+0000","last_change":"2023-12-12T09:52:08.035067+0000","last_active":"2023-12-12T09:52:12.104135+0000","last_peered":"2023-12-12T09:52:12.104135+0000","last_clean":"2023-12-12T09:52:12.104135+0000","last_became_active":"2023-12-12T09:52:08.034803+0000","last_became_peered":"2023-12-12T09:52:08.034803+0000","last_unstale":"2023-12-12T09:52:12.104135+0000","last_undegraded":"2023-12-12T09:52:12.104135+0000","last_fullsized":"2023-12-12T09:52:12.104135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:22.978364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:27.255184+0000","last_change":"2023-12-12T09:52:06.998675+0000","last_active":"2023-12-12T09:52:27.255184+0000","last_peered":"2023-12-12T09:52:27.255184+0000","last_clean":"2023-12-12T09:52:27.255184+0000","last_became_active":"2023-12-12T09:52:06.998364+0000","last_became_peered":"2023-12-12T09:52:06.998364+0000","last_unstale":"2023-12-12T09:52:27.255184+0000","last_undegraded":"2023-12-12T09:52:27.255184+0000","last_fullsized":"2023-12-12T09:52:27.255184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3839,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283450+0000","last_change":"2023-12-12T09:52:12.086437+0000","last_active":"2023-12-12T09:52:13.283450+0000","last_peered":"2023-12-12T09:52:13.283450+0000","last_clean":"2023-12-12T09:52:13.283450+0000","last_became_active":"2023-12-12T09:52:12.083068+0000","last_became_peered":"2023-12-12T09:52:12.083068+0000","last_unstale":"2023-12-12T09:52:13.283450+0000","last_undegraded":"2023-12-12T09:52:13.283450+0000","last_fullsized":"2023-12-12T09:52:13.283450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066756+0000","last_change":"2023-12-12T09:52:08.030198+0000","last_active":"2023-12-12T09:52:12.066756+0000","last_peered":"2023-12-12T09:52:12.066756+0000","last_clean":"2023-12-12T09:52:12.066756+0000","last_became_active":"2023-12-12T09:52:08.027733+0000","last_became_peered":"2023-12-12T09:52:08.027733+0000","last_unstale":"2023-12-12T09:52:12.066756+0000","last_undegraded":"2023-12-12T09:52:12.066756+0000","last_fullsized":"2023-12-12T09:52:12.066756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:08.788149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056359+0000","last_change":"2023-12-12T09:52:07.004177+0000","last_active":"2023-12-12T09:52:12.056359+0000","last_peered":"2023-12-12T09:52:12.056359+0000","last_clean":"2023-12-12T09:52:12.056359+0000","last_became_active":"2023-12-12T09:52:07.003836+0000","last_became_peered":"2023-12-12T09:52:07.003836+0000","last_unstale":"2023-12-12T09:52:12.056359+0000","last_undegraded":"2023-12-12T09:52:12.056359+0000","last_fullsized":"2023-12-12T09:52:12.056359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284483+0000","last_change":"2023-12-12T09:52:12.081391+0000","last_active":"2023-12-12T09:52:13.284483+0000","last_peered":"2023-12-12T09:52:13.284483+0000","last_clean":"2023-12-12T09:52:13.284483+0000","last_became_active":"2023-12-12T09:52:12.080949+0000","last_became_peered":"2023-12-12T09:52:12.080949+0000","last_unstale":"2023-12-12T09:52:13.284483+0000","last_undegraded":"2023-12-12T09:52:13.284483+0000","last_fullsized":"2023-12-12T09:52:13.284483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:39.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070164+0000","last_change":"2023-12-12T09:52:08.023308+0000","last_active":"2023-12-12T09:52:12.070164+0000","last_peered":"2023-12-12T09:52:12.070164+0000","last_clean":"2023-12-12T09:52:12.070164+0000","last_became_active":"2023-12-12T09:52:08.023048+0000","last_became_peered":"2023-12-12T09:52:08.023048+0000","last_unstale":"2023-12-12T09:52:12.070164+0000","last_undegraded":"2023-12-12T09:52:12.070164+0000","last_fullsized":"2023-12-12T09:52:12.070164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075599+0000","last_change":"2023-12-12T09:52:07.001746+0000","last_active":"2023-12-12T09:52:12.075599+0000","last_peered":"2023-12-12T09:52:12.075599+0000","last_clean":"2023-12-12T09:52:12.075599+0000","last_became_active":"2023-12-12T09:52:07.001424+0000","last_became_peered":"2023-12-12T09:52:07.001424+0000","last_unstale":"2023-12-12T09:52:12.075599+0000","last_undegraded":"2023-12-12T09:52:12.075599+0000","last_fullsized":"2023-12-12T09:52:12.075599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:54.297647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282877+0000","last_change":"2023-12-12T09:52:12.077880+0000","last_active":"2023-12-12T09:52:13.282877+0000","last_peered":"2023-12-12T09:52:13.282877+0000","last_clean":"2023-12-12T09:52:13.282877+0000","last_became_active":"2023-12-12T09:52:12.077365+0000","last_became_peered":"2023-12-12T09:52:12.077365+0000","last_unstale":"2023-12-12T09:52:13.282877+0000","last_undegraded":"2023-12-12T09:52:13.282877+0000","last_fullsized":"2023-12-12T09:52:13.282877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:43:05.307563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075525+0000","last_change":"2023-12-12T09:52:08.029359+0000","last_active":"2023-12-12T09:52:12.075525+0000","last_peered":"2023-12-12T09:52:12.075525+0000","last_clean":"2023-12-12T09:52:12.075525+0000","last_became_active":"2023-12-12T09:52:08.028997+0000","last_became_peered":"2023-12-12T09:52:08.028997+0000","last_unstale":"2023-12-12T09:52:12.075525+0000","last_undegraded":"2023-12-12T09:52:12.075525+0000","last_fullsized":"2023-12-12T09:52:12.075525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:22.978364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.052142+0000","last_change":"2023-12-12T09:52:07.006953+0000","last_active":"2023-12-12T09:52:12.052142+0000","last_peered":"2023-12-12T09:52:12.052142+0000","last_clean":"2023-12-12T09:52:12.052142+0000","last_became_active":"2023-12-12T09:52:07.006666+0000","last_became_peered":"2023-12-12T09:52:07.006666+0000","last_unstale":"2023-12-12T09:52:12.052142+0000","last_undegraded":"2023-12-12T09:52:12.052142+0000","last_fullsized":"2023-12-12T09:52:12.052142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:26:34.669025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283756+0000","last_change":"2023-12-12T09:52:12.104433+0000","last_active":"2023-12-12T09:52:13.283756+0000","last_peered":"2023-12-12T09:52:13.283756+0000","last_clean":"2023-12-12T09:52:13.283756+0000","last_became_active":"2023-12-12T09:52:12.104126+0000","last_became_peered":"2023-12-12T09:52:12.104126+0000","last_unstale":"2023-12-12T09:52:13.283756+0000","last_undegraded":"2023-12-12T09:52:13.283756+0000","last_fullsized":"2023-12-12T09:52:13.283756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:25:03.440488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.289874+0000","last_change":"2023-12-12T09:52:12.111376+0000","last_active":"2023-12-12T09:52:13.289874+0000","last_peered":"2023-12-12T09:52:13.289874+0000","last_clean":"2023-12-12T09:52:13.289874+0000","last_became_active":"2023-12-12T09:52:12.110863+0000","last_became_peered":"2023-12-12T09:52:12.110863+0000","last_unstale":"2023-12-12T09:52:13.289874+0000","last_undegraded":"2023-12-12T09:52:13.289874+0000","last_fullsized":"2023-12-12T09:52:13.289874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:55.497789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055671+0000","last_change":"2023-12-12T09:52:08.024511+0000","last_active":"2023-12-12T09:52:12.055671+0000","last_peered":"2023-12-12T09:52:12.055671+0000","last_clean":"2023-12-12T09:52:12.055671+0000","last_became_active":"2023-12-12T09:52:08.024258+0000","last_became_peered":"2023-12-12T09:52:08.024258+0000","last_unstale":"2023-12-12T09:52:12.055671+0000","last_undegraded":"2023-12-12T09:52:12.055671+0000","last_fullsized":"2023-12-12T09:52:12.055671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074250+0000","last_change":"2023-12-12T09:52:07.000641+0000","last_active":"2023-12-12T09:52:12.074250+0000","last_peered":"2023-12-12T09:52:12.074250+0000","last_clean":"2023-12-12T09:52:12.074250+0000","last_became_active":"2023-12-12T09:52:07.000245+0000","last_became_peered":"2023-12-12T09:52:07.000245+0000","last_unstale":"2023-12-12T09:52:12.074250+0000","last_undegraded":"2023-12-12T09:52:12.074250+0000","last_fullsized":"2023-12-12T09:52:12.074250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:46.546348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.056067+0000","last_change":"2023-12-12T09:52:12.078229+0000","last_active":"2023-12-12T09:52:13.056067+0000","last_peered":"2023-12-12T09:52:13.056067+0000","last_clean":"2023-12-12T09:52:13.056067+0000","last_became_active":"2023-12-12T09:52:12.077764+0000","last_became_peered":"2023-12-12T09:52:12.077764+0000","last_unstale":"2023-12-12T09:52:13.056067+0000","last_undegraded":"2023-12-12T09:52:13.056067+0000","last_fullsized":"2023-12-12T09:52:13.056067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:21:02.014413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059639+0000","last_change":"2023-12-12T09:52:08.028827+0000","last_active":"2023-12-12T09:52:12.059639+0000","last_peered":"2023-12-12T09:52:12.059639+0000","last_clean":"2023-12-12T09:52:12.059639+0000","last_became_active":"2023-12-12T09:52:08.024860+0000","last_became_peered":"2023-12-12T09:52:08.024860+0000","last_unstale":"2023-12-12T09:52:12.059639+0000","last_undegraded":"2023-12-12T09:52:12.059639+0000","last_fullsized":"2023-12-12T09:52:12.059639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054975+0000","last_change":"2023-12-12T09:52:06.999381+0000","last_active":"2023-12-12T09:52:12.054975+0000","last_peered":"2023-12-12T09:52:12.054975+0000","last_clean":"2023-12-12T09:52:12.054975+0000","last_became_active":"2023-12-12T09:52:06.999099+0000","last_became_peered":"2023-12-12T09:52:06.999099+0000","last_unstale":"2023-12-12T09:52:12.054975+0000","last_undegraded":"2023-12-12T09:52:12.054975+0000","last_fullsized":"2023-12-12T09:52:12.054975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050899+0000","last_change":"2023-12-12T09:52:07.003879+0000","last_active":"2023-12-12T09:52:12.050899+0000","last_peered":"2023-12-12T09:52:12.050899+0000","last_clean":"2023-12-12T09:52:12.050899+0000","last_became_active":"2023-12-12T09:52:07.002760+0000","last_became_peered":"2023-12-12T09:52:07.002760+0000","last_unstale":"2023-12-12T09:52:12.050899+0000","last_undegraded":"2023-12-12T09:52:12.050899+0000","last_fullsized":"2023-12-12T09:52:12.050899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:07.777932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068247+0000","last_change":"2023-12-12T09:52:08.026740+0000","last_active":"2023-12-12T09:52:12.068247+0000","last_peered":"2023-12-12T09:52:12.068247+0000","last_clean":"2023-12-12T09:52:12.068247+0000","last_became_active":"2023-12-12T09:52:08.024698+0000","last_became_peered":"2023-12-12T09:52:08.024698+0000","last_unstale":"2023-12-12T09:52:12.068247+0000","last_undegraded":"2023-12-12T09:52:12.068247+0000","last_fullsized":"2023-12-12T09:52:12.068247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284952+0000","last_change":"2023-12-12T09:52:12.078748+0000","last_active":"2023-12-12T09:52:13.284952+0000","last_peered":"2023-12-12T09:52:13.284952+0000","last_clean":"2023-12-12T09:52:13.284952+0000","last_became_active":"2023-12-12T09:52:12.077889+0000","last_became_peered":"2023-12-12T09:52:12.077889+0000","last_unstale":"2023-12-12T09:52:13.284952+0000","last_undegraded":"2023-12-12T09:52:13.284952+0000","last_fullsized":"2023-12-12T09:52:13.284952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:05.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054823+0000","last_change":"2023-12-12T09:52:07.008787+0000","last_active":"2023-12-12T09:52:12.054823+0000","last_peered":"2023-12-12T09:52:12.054823+0000","last_clean":"2023-12-12T09:52:12.054823+0000","last_became_active":"2023-12-12T09:52:07.008455+0000","last_became_peered":"2023-12-12T09:52:07.008455+0000","last_unstale":"2023-12-12T09:52:12.054823+0000","last_undegraded":"2023-12-12T09:52:12.054823+0000","last_fullsized":"2023-12-12T09:52:12.054823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.061074+0000","last_change":"2023-12-12T09:52:08.022255+0000","last_active":"2023-12-12T09:52:12.061074+0000","last_peered":"2023-12-12T09:52:12.061074+0000","last_clean":"2023-12-12T09:52:12.061074+0000","last_became_active":"2023-12-12T09:52:08.020240+0000","last_became_peered":"2023-12-12T09:52:08.020240+0000","last_unstale":"2023-12-12T09:52:12.061074+0000","last_undegraded":"2023-12-12T09:52:12.061074+0000","last_fullsized":"2023-12-12T09:52:12.061074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:22.201340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.286721+0000","last_change":"2023-12-12T09:52:12.081878+0000","last_active":"2023-12-12T09:52:13.286721+0000","last_peered":"2023-12-12T09:52:13.286721+0000","last_clean":"2023-12-12T09:52:13.286721+0000","last_became_active":"2023-12-12T09:52:12.081502+0000","last_became_peered":"2023-12-12T09:52:12.081502+0000","last_unstale":"2023-12-12T09:52:13.286721+0000","last_undegraded":"2023-12-12T09:52:13.286721+0000","last_fullsized":"2023-12-12T09:52:13.286721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:18:41.559574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050317+0000","last_change":"2023-12-12T09:52:08.021237+0000","last_active":"2023-12-12T09:52:12.050317+0000","last_peered":"2023-12-12T09:52:12.050317+0000","last_clean":"2023-12-12T09:52:12.050317+0000","last_became_active":"2023-12-12T09:52:08.019519+0000","last_became_peered":"2023-12-12T09:52:08.019519+0000","last_unstale":"2023-12-12T09:52:12.050317+0000","last_undegraded":"2023-12-12T09:52:12.050317+0000","last_fullsized":"2023-12-12T09:52:12.050317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:01.082803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054863+0000","last_change":"2023-12-12T09:52:08.019363+0000","last_active":"2023-12-12T09:52:12.054863+0000","last_peered":"2023-12-12T09:52:12.054863+0000","last_clean":"2023-12-12T09:52:12.054863+0000","last_became_active":"2023-12-12T09:52:08.012574+0000","last_became_peered":"2023-12-12T09:52:08.012574+0000","last_unstale":"2023-12-12T09:52:12.054863+0000","last_undegraded":"2023-12-12T09:52:12.054863+0000","last_fullsized":"2023-12-12T09:52:12.054863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:08.788149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074382+0000","last_change":"2023-12-12T09:52:08.030165+0000","last_active":"2023-12-12T09:52:12.074382+0000","last_peered":"2023-12-12T09:52:12.074382+0000","last_clean":"2023-12-12T09:52:12.074382+0000","last_became_active":"2023-12-12T09:52:08.029626+0000","last_became_peered":"2023-12-12T09:52:08.029626+0000","last_unstale":"2023-12-12T09:52:12.074382+0000","last_undegraded":"2023-12-12T09:52:12.074382+0000","last_fullsized":"2023-12-12T09:52:12.074382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068155+0000","last_change":"2023-12-12T09:52:08.040599+0000","last_active":"2023-12-12T09:52:12.068155+0000","last_peered":"2023-12-12T09:52:12.068155+0000","last_clean":"2023-12-12T09:52:12.068155+0000","last_became_active":"2023-12-12T09:52:08.039875+0000","last_became_peered":"2023-12-12T09:52:08.039875+0000","last_unstale":"2023-12-12T09:52:12.068155+0000","last_undegraded":"2023-12-12T09:52:12.068155+0000","last_fullsized":"2023-12-12T09:52:12.068155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060010+0000","last_change":"2023-12-12T09:52:08.032398+0000","last_active":"2023-12-12T09:52:12.060010+0000","last_peered":"2023-12-12T09:52:12.060010+0000","last_clean":"2023-12-12T09:52:12.060010+0000","last_became_active":"2023-12-12T09:52:08.031194+0000","last_became_peered":"2023-12-12T09:52:08.031194+0000","last_unstale":"2023-12-12T09:52:12.060010+0000","last_undegraded":"2023-12-12T09:52:12.060010+0000","last_fullsized":"2023-12-12T09:52:12.060010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056859+0000","last_change":"2023-12-12T09:52:08.023831+0000","last_active":"2023-12-12T09:52:12.056859+0000","last_peered":"2023-12-12T09:52:12.056859+0000","last_clean":"2023-12-12T09:52:12.056859+0000","last_became_active":"2023-12-12T09:52:08.023532+0000","last_became_peered":"2023-12-12T09:52:08.023532+0000","last_unstale":"2023-12-12T09:52:12.056859+0000","last_undegraded":"2023-12-12T09:52:12.056859+0000","last_fullsized":"2023-12-12T09:52:12.056859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054670+0000","last_change":"2023-12-12T09:52:08.028457+0000","last_active":"2023-12-12T09:52:12.054670+0000","last_peered":"2023-12-12T09:52:12.054670+0000","last_clean":"2023-12-12T09:52:12.054670+0000","last_became_active":"2023-12-12T09:52:08.028172+0000","last_became_peered":"2023-12-12T09:52:08.028172+0000","last_unstale":"2023-12-12T09:52:12.054670+0000","last_undegraded":"2023-12-12T09:52:12.054670+0000","last_fullsized":"2023-12-12T09:52:12.054670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.063395+0000","last_change":"2023-12-12T09:52:08.033841+0000","last_active":"2023-12-12T09:52:12.063395+0000","last_peered":"2023-12-12T09:52:12.063395+0000","last_clean":"2023-12-12T09:52:12.063395+0000","last_became_active":"2023-12-12T09:52:08.030880+0000","last_became_peered":"2023-12-12T09:52:08.030880+0000","last_unstale":"2023-12-12T09:52:12.063395+0000","last_undegraded":"2023-12-12T09:52:12.063395+0000","last_fullsized":"2023-12-12T09:52:12.063395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074138+0000","last_change":"2023-12-12T09:52:08.032051+0000","last_active":"2023-12-12T09:52:12.074138+0000","last_peered":"2023-12-12T09:52:12.074138+0000","last_clean":"2023-12-12T09:52:12.074138+0000","last_became_active":"2023-12-12T09:52:08.031785+0000","last_became_peered":"2023-12-12T09:52:08.031785+0000","last_unstale":"2023-12-12T09:52:12.074138+0000","last_undegraded":"2023-12-12T09:52:12.074138+0000","last_fullsized":"2023-12-12T09:52:12.074138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054886+0000","last_change":"2023-12-12T09:52:08.031522+0000","last_active":"2023-12-12T09:52:12.054886+0000","last_peered":"2023-12-12T09:52:12.054886+0000","last_clean":"2023-12-12T09:52:12.054886+0000","last_became_active":"2023-12-12T09:52:08.031262+0000","last_became_peered":"2023-12-12T09:52:08.031262+0000","last_unstale":"2023-12-12T09:52:12.054886+0000","last_undegraded":"2023-12-12T09:52:12.054886+0000","last_fullsized":"2023-12-12T09:52:12.054886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074154+0000","last_change":"2023-12-12T09:52:08.024395+0000","last_active":"2023-12-12T09:52:12.074154+0000","last_peered":"2023-12-12T09:52:12.074154+0000","last_clean":"2023-12-12T09:52:12.074154+0000","last_became_active":"2023-12-12T09:52:08.024105+0000","last_became_peered":"2023-12-12T09:52:08.024105+0000","last_unstale":"2023-12-12T09:52:12.074154+0000","last_undegraded":"2023-12-12T09:52:12.074154+0000","last_fullsized":"2023-12-12T09:52:12.074154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.760548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.069490+0000","last_change":"2023-12-12T09:52:08.037425+0000","last_active":"2023-12-12T09:52:12.069490+0000","last_peered":"2023-12-12T09:52:12.069490+0000","last_clean":"2023-12-12T09:52:12.069490+0000","last_became_active":"2023-12-12T09:52:08.035756+0000","last_became_peered":"2023-12-12T09:52:08.035756+0000","last_unstale":"2023-12-12T09:52:12.069490+0000","last_undegraded":"2023-12-12T09:52:12.069490+0000","last_fullsized":"2023-12-12T09:52:12.069490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:01.082803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050903+0000","last_change":"2023-12-12T09:52:08.031988+0000","last_active":"2023-12-12T09:52:12.050903+0000","last_peered":"2023-12-12T09:52:12.050903+0000","last_clean":"2023-12-12T09:52:12.050903+0000","last_became_active":"2023-12-12T09:52:08.031525+0000","last_became_peered":"2023-12-12T09:52:08.031525+0000","last_unstale":"2023-12-12T09:52:12.050903+0000","last_undegraded":"2023-12-12T09:52:12.050903+0000","last_fullsized":"2023-12-12T09:52:12.050903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104392+0000","last_change":"2023-12-12T09:52:08.034992+0000","last_active":"2023-12-12T09:52:12.104392+0000","last_peered":"2023-12-12T09:52:12.104392+0000","last_clean":"2023-12-12T09:52:12.104392+0000","last_became_active":"2023-12-12T09:52:08.034424+0000","last_became_peered":"2023-12-12T09:52:08.034424+0000","last_unstale":"2023-12-12T09:52:12.104392+0000","last_undegraded":"2023-12-12T09:52:12.104392+0000","last_fullsized":"2023-12-12T09:52:12.104392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:24.493525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053756+0000","last_change":"2023-12-12T09:52:08.021603+0000","last_active":"2023-12-12T09:52:12.053756+0000","last_peered":"2023-12-12T09:52:12.053756+0000","last_clean":"2023-12-12T09:52:12.053756+0000","last_became_active":"2023-12-12T09:52:08.021253+0000","last_became_peered":"2023-12-12T09:52:08.021253+0000","last_unstale":"2023-12-12T09:52:12.053756+0000","last_undegraded":"2023-12-12T09:52:12.053756+0000","last_fullsized":"2023-12-12T09:52:12.053756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:45.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075822+0000","last_change":"2023-12-12T09:52:08.036026+0000","last_active":"2023-12-12T09:52:12.075822+0000","last_peered":"2023-12-12T09:52:12.075822+0000","last_clean":"2023-12-12T09:52:12.075822+0000","last_became_active":"2023-12-12T09:52:08.035390+0000","last_became_peered":"2023-12-12T09:52:08.035390+0000","last_unstale":"2023-12-12T09:52:12.075822+0000","last_undegraded":"2023-12-12T09:52:12.075822+0000","last_fullsized":"2023-12-12T09:52:12.075822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:51.434174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.073460+0000","last_change":"2023-12-12T09:52:08.013438+0000","last_active":"2023-12-12T09:52:12.073460+0000","last_peered":"2023-12-12T09:52:12.073460+0000","last_clean":"2023-12-12T09:52:12.073460+0000","last_became_active":"2023-12-12T09:52:08.013002+0000","last_became_peered":"2023-12-12T09:52:08.013002+0000","last_unstale":"2023-12-12T09:52:12.073460+0000","last_undegraded":"2023-12-12T09:52:12.073460+0000","last_fullsized":"2023-12-12T09:52:12.073460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055788+0000","last_change":"2023-12-12T09:52:08.032162+0000","last_active":"2023-12-12T09:52:12.055788+0000","last_peered":"2023-12-12T09:52:12.055788+0000","last_clean":"2023-12-12T09:52:12.055788+0000","last_became_active":"2023-12-12T09:52:08.031735+0000","last_became_peered":"2023-12-12T09:52:08.031735+0000","last_unstale":"2023-12-12T09:52:12.055788+0000","last_undegraded":"2023-12-12T09:52:12.055788+0000","last_fullsized":"2023-12-12T09:52:12.055788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070823+0000","last_change":"2023-12-12T09:52:08.024576+0000","last_active":"2023-12-12T09:52:12.070823+0000","last_peered":"2023-12-12T09:52:12.070823+0000","last_clean":"2023-12-12T09:52:12.070823+0000","last_became_active":"2023-12-12T09:52:08.024277+0000","last_became_peered":"2023-12-12T09:52:08.024277+0000","last_unstale":"2023-12-12T09:52:12.070823+0000","last_undegraded":"2023-12-12T09:52:12.070823+0000","last_fullsized":"2023-12-12T09:52:12.070823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:48.900772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054635+0000","last_change":"2023-12-12T09:52:08.029262+0000","last_active":"2023-12-12T09:52:12.054635+0000","last_peered":"2023-12-12T09:52:12.054635+0000","last_clean":"2023-12-12T09:52:12.054635+0000","last_became_active":"2023-12-12T09:52:08.028949+0000","last_became_peered":"2023-12-12T09:52:08.028949+0000","last_unstale":"2023-12-12T09:52:12.054635+0000","last_undegraded":"2023-12-12T09:52:12.054635+0000","last_fullsized":"2023-12-12T09:52:12.054635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066542+0000","last_change":"2023-12-12T09:52:08.024637+0000","last_active":"2023-12-12T09:52:12.066542+0000","last_peered":"2023-12-12T09:52:12.066542+0000","last_clean":"2023-12-12T09:52:12.066542+0000","last_became_active":"2023-12-12T09:52:08.023754+0000","last_became_peered":"2023-12-12T09:52:08.023754+0000","last_unstale":"2023-12-12T09:52:12.066542+0000","last_undegraded":"2023-12-12T09:52:12.066542+0000","last_fullsized":"2023-12-12T09:52:12.066542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:45.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055913+0000","last_change":"2023-12-12T09:52:08.024613+0000","last_active":"2023-12-12T09:52:12.055913+0000","last_peered":"2023-12-12T09:52:12.055913+0000","last_clean":"2023-12-12T09:52:12.055913+0000","last_became_active":"2023-12-12T09:52:08.024257+0000","last_became_peered":"2023-12-12T09:52:08.024257+0000","last_unstale":"2023-12-12T09:52:12.055913+0000","last_undegraded":"2023-12-12T09:52:12.055913+0000","last_fullsized":"2023-12-12T09:52:12.055913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054417+0000","last_change":"2023-12-12T09:52:08.023097+0000","last_active":"2023-12-12T09:52:12.054417+0000","last_peered":"2023-12-12T09:52:12.054417+0000","last_clean":"2023-12-12T09:52:12.054417+0000","last_became_active":"2023-12-12T09:52:08.022866+0000","last_became_peered":"2023-12-12T09:52:08.022866+0000","last_unstale":"2023-12-12T09:52:12.054417+0000","last_undegraded":"2023-12-12T09:52:12.054417+0000","last_fullsized":"2023-12-12T09:52:12.054417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104813+0000","last_change":"2023-12-12T09:52:08.039672+0000","last_active":"2023-12-12T09:52:12.104813+0000","last_peered":"2023-12-12T09:52:12.104813+0000","last_clean":"2023-12-12T09:52:12.104813+0000","last_became_active":"2023-12-12T09:52:08.039145+0000","last_became_peered":"2023-12-12T09:52:08.039145+0000","last_unstale":"2023-12-12T09:52:12.104813+0000","last_undegraded":"2023-12-12T09:52:12.104813+0000","last_fullsized":"2023-12-12T09:52:12.104813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:47.907544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059506+0000","last_change":"2023-12-12T09:52:08.030292+0000","last_active":"2023-12-12T09:52:12.059506+0000","last_peered":"2023-12-12T09:52:12.059506+0000","last_clean":"2023-12-12T09:52:12.059506+0000","last_became_active":"2023-12-12T09:52:08.029117+0000","last_became_peered":"2023-12-12T09:52:08.029117+0000","last_unstale":"2023-12-12T09:52:12.059506+0000","last_undegraded":"2023-12-12T09:52:12.059506+0000","last_fullsized":"2023-12-12T09:52:12.059506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103702+0000","last_change":"2023-12-12T09:52:08.040587+0000","last_active":"2023-12-12T09:52:12.103702+0000","last_peered":"2023-12-12T09:52:12.103702+0000","last_clean":"2023-12-12T09:52:12.103702+0000","last_became_active":"2023-12-12T09:52:08.040204+0000","last_became_peered":"2023-12-12T09:52:08.040204+0000","last_unstale":"2023-12-12T09:52:12.103702+0000","last_undegraded":"2023-12-12T09:52:12.103702+0000","last_fullsized":"2023-12-12T09:52:12.103702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.073636+0000","last_change":"2023-12-12T09:52:08.015126+0000","last_active":"2023-12-12T09:52:12.073636+0000","last_peered":"2023-12-12T09:52:12.073636+0000","last_clean":"2023-12-12T09:52:12.073636+0000","last_became_active":"2023-12-12T09:52:08.014817+0000","last_became_peered":"2023-12-12T09:52:08.014817+0000","last_unstale":"2023-12-12T09:52:12.073636+0000","last_undegraded":"2023-12-12T09:52:12.073636+0000","last_fullsized":"2023-12-12T09:52:12.073636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103966+0000","last_change":"2023-12-12T09:52:08.034136+0000","last_active":"2023-12-12T09:52:12.103966+0000","last_peered":"2023-12-12T09:52:12.103966+0000","last_clean":"2023-12-12T09:52:12.103966+0000","last_became_active":"2023-12-12T09:52:08.033810+0000","last_became_peered":"2023-12-12T09:52:08.033810+0000","last_unstale":"2023-12-12T09:52:12.103966+0000","last_undegraded":"2023-12-12T09:52:12.103966+0000","last_fullsized":"2023-12-12T09:52:12.103966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.950798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074787+0000","last_change":"2023-12-12T09:52:08.032774+0000","last_active":"2023-12-12T09:52:12.074787+0000","last_peered":"2023-12-12T09:52:12.074787+0000","last_clean":"2023-12-12T09:52:12.074787+0000","last_became_active":"2023-12-12T09:52:08.030758+0000","last_became_peered":"2023-12-12T09:52:08.030758+0000","last_unstale":"2023-12-12T09:52:12.074787+0000","last_undegraded":"2023-12-12T09:52:12.074787+0000","last_fullsized":"2023-12-12T09:52:12.074787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065632+0000","last_change":"2023-12-12T09:52:08.032385+0000","last_active":"2023-12-12T09:52:12.065632+0000","last_peered":"2023-12-12T09:52:12.065632+0000","last_clean":"2023-12-12T09:52:12.065632+0000","last_became_active":"2023-12-12T09:52:08.032067+0000","last_became_peered":"2023-12-12T09:52:08.032067+0000","last_unstale":"2023-12-12T09:52:12.065632+0000","last_undegraded":"2023-12-12T09:52:12.065632+0000","last_fullsized":"2023-12-12T09:52:12.065632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056995+0000","last_change":"2023-12-12T09:52:08.028105+0000","last_active":"2023-12-12T09:52:12.056995+0000","last_peered":"2023-12-12T09:52:12.056995+0000","last_clean":"2023-12-12T09:52:12.056995+0000","last_became_active":"2023-12-12T09:52:08.027786+0000","last_became_peered":"2023-12-12T09:52:08.027786+0000","last_unstale":"2023-12-12T09:52:12.056995+0000","last_undegraded":"2023-12-12T09:52:12.056995+0000","last_fullsized":"2023-12-12T09:52:12.056995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104765+0000","last_change":"2023-12-12T09:52:08.034061+0000","last_active":"2023-12-12T09:52:12.104765+0000","last_peered":"2023-12-12T09:52:12.104765+0000","last_clean":"2023-12-12T09:52:12.104765+0000","last_became_active":"2023-12-12T09:52:08.033745+0000","last_became_peered":"2023-12-12T09:52:08.033745+0000","last_unstale":"2023-12-12T09:52:12.104765+0000","last_undegraded":"2023-12-12T09:52:12.104765+0000","last_fullsized":"2023-12-12T09:52:12.104765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:51.434174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4637,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":98899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":98933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:52:38.303 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-12T09:52:38.303 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-12T09:52:38.303 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-12T09:52:38.304 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-12T09:52:38.304 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-12T09:52:38.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.546+0000 7efe38200640 1 Processor -- start 2023-12-12T09:52:38.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.547+0000 7efe38200640 1 -- start start 2023-12-12T09:52:38.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe38200640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efe301607f0 0x7efe300ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe38200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe300cecb0 0x7efe300cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe38200640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300b6fe0 con 0x7efe300cff00 2023-12-12T09:52:38.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe38200640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300d1be0 con 0x7efe301607f0 2023-12-12T09:52:38.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe38200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300d0d50 con 0x7efe300cecb0 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe35f9c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efe301607f0 0x7efe300ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe35f9c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efe301607f0 0x7efe300ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40246/0 (socket says 172.21.15.72:40246) 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe3679d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7efe300cff00 0x7efe30160ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52368/0 (socket says 172.21.15.72:52368) 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.548+0000 7efe35f9c640 1 -- 172.21.15.72:0/2495060208 learned_addr learned my addr 172.21.15.72:0/2495060208 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.549+0000 7efe3579b640 1 --2- 172.21.15.72:0/2495060208 >> v2:172.21.15.113:3300/0 conn(0x7efe300cecb0 0x7efe300cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.549+0000 7efe35f9c640 1 -- 172.21.15.72:0/2495060208 >> v2:172.21.15.113:3300/0 conn(0x7efe300cecb0 msgr2=0x7efe300cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.549+0000 7efe35f9c640 1 --2- 172.21.15.72:0/2495060208 >> v2:172.21.15.113:3300/0 conn(0x7efe300cecb0 0x7efe300cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.549+0000 7efe35f9c640 1 -- 172.21.15.72:0/2495060208 >> v1:172.21.15.72:6789/0 conn(0x7efe300cff00 legacy=0x7efe30160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.549+0000 7efe35f9c640 1 -- 172.21.15.72:0/2495060208 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300d0f60 con 0x7efe301607f0 2023-12-12T09:52:38.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.549+0000 7efe3579b640 1 --2- 172.21.15.72:0/2495060208 >> v2:172.21.15.113:3300/0 conn(0x7efe300cecb0 0x7efe300cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:38.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.550+0000 7efe35f9c640 1 --2- 172.21.15.72:0/2495060208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe301607f0 0x7efe300ce770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efe30073bb0 tx=0x7efe2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d598f46d03625e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:38.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.550+0000 7efe34f9a640 1 -- 172.21.15.72:0/2495060208 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c031870 con 0x7efe301607f0 2023-12-12T09:52:38.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.550+0000 7efe34f9a640 1 -- 172.21.15.72:0/2495060208 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c0319d0 con 0x7efe301607f0 2023-12-12T09:52:38.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.550+0000 7efe34f9a640 1 -- 172.21.15.72:0/2495060208 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c031d60 con 0x7efe301607f0 2023-12-12T09:52:38.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.551+0000 7efe38200640 1 -- 172.21.15.72:0/2495060208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe301607f0 msgr2=0x7efe300ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.551+0000 7efe38200640 1 --2- 172.21.15.72:0/2495060208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe301607f0 0x7efe300ce770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efe30073bb0 tx=0x7efe2c02b910 comp rx=0 tx=0).stop 2023-12-12T09:52:38.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.551+0000 7efe38200640 1 -- 172.21.15.72:0/2495060208 shutdown_connections 2023-12-12T09:52:38.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.551+0000 7efe38200640 1 --2- 172.21.15.72:0/2495060208 >> v2:172.21.15.113:3300/0 conn(0x7efe300cecb0 0x7efe300cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.551+0000 7efe38200640 1 --2- 172.21.15.72:0/2495060208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe301607f0 0x7efe300ce770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.551+0000 7efe38200640 1 -- 172.21.15.72:0/2495060208 >> 172.21.15.72:0/2495060208 conn(0x7efe3009c450 msgr2=0x7efe300d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:38.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.551+0000 7efe38200640 1 -- 172.21.15.72:0/2495060208 shutdown_connections 2023-12-12T09:52:38.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.551+0000 7efe38200640 1 -- 172.21.15.72:0/2495060208 wait complete. 2023-12-12T09:52:38.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.552+0000 7efe38200640 1 Processor -- start 2023-12-12T09:52:38.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.553+0000 7efe38200640 1 -- start start 2023-12-12T09:52:38.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.553+0000 7efe38200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe300cecb0 0x7efe301f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.553+0000 7efe38200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe300cff00 0x7efe30157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe38200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe301607f0 0x7efe301f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe38200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe301f3d70 con 0x7efe300cff00 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe38200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe301f3b70 con 0x7efe300cecb0 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe38200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3015a0d0 con 0x7efe301607f0 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe300cff00 0x7efe30157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe300cff00 0x7efe30157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40248/0 (socket says 172.21.15.72:40248) 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 -- 172.21.15.72:0/2351560425 learned_addr learned my addr 172.21.15.72:0/2351560425 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe35f9c640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe300cecb0 0x7efe301f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 -- 172.21.15.72:0/2351560425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe301607f0 msgr2=0x7efe301f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe301607f0 0x7efe301f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 -- 172.21.15.72:0/2351560425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe300cecb0 msgr2=0x7efe301f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe300cecb0 0x7efe301f3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 -- 172.21.15.72:0/2351560425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c0094d0 con 0x7efe300cff00 2023-12-12T09:52:38.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.554+0000 7efe3579b640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe300cff00 0x7efe30157e80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c820 tx=0x7efe2000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:38.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.555+0000 7efe267fc640 1 -- 172.21.15.72:0/2351560425 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20007a30 con 0x7efe300cff00 2023-12-12T09:52:38.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.555+0000 7efe267fc640 1 -- 172.21.15.72:0/2351560425 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe20007b90 con 0x7efe300cff00 2023-12-12T09:52:38.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.555+0000 7efe267fc640 1 -- 172.21.15.72:0/2351560425 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2000ce10 con 0x7efe300cff00 2023-12-12T09:52:38.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.555+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3015a3b0 con 0x7efe300cff00 2023-12-12T09:52:38.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.555+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe300b58f0 con 0x7efe300cff00 2023-12-12T09:52:38.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.556+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe301f44e0 con 0x7efe300cff00 2023-12-12T09:52:38.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.557+0000 7efe267fc640 1 -- 172.21.15.72:0/2351560425 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe200113f0 con 0x7efe300cff00 2023-12-12T09:52:38.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.558+0000 7efe267fc640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe1406ae40 0x7efe1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:38.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.558+0000 7efe267fc640 1 -- 172.21.15.72:0/2351560425 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efe20089a30 con 0x7efe300cff00 2023-12-12T09:52:38.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.558+0000 7efe35f9c640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe1406ae40 0x7efe1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:38.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.559+0000 7efe35f9c640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe1406ae40 0x7efe1406d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe2c036000 tx=0x7efe2c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:38.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.563+0000 7efe267fc640 1 -- 172.21.15.72:0/2351560425 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe301f44e0 con 0x7efe300cff00 2023-12-12T09:52:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.782+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efe301f44e0 con 0x7efe300cff00 2023-12-12T09:52:38.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.791+0000 7efe267fc640 1 -- 172.21.15.72:0/2351560425 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171028 (secure 0 0 0) 0x7efe301f44e0 con 0x7efe300cff00 2023-12-12T09:52:38.795 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe1406ae40 msgr2=0x7efe1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe1406ae40 0x7efe1406d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe2c036000 tx=0x7efe2c03c040 comp rx=0 tx=0).stop 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe300cff00 msgr2=0x7efe30157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe300cff00 0x7efe30157e80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c820 tx=0x7efe2000cce0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 shutdown_connections 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe1406ae40 0x7efe1406d300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe301607f0 0x7efe301f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe300cff00 0x7efe30157e80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 --2- 172.21.15.72:0/2351560425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe300cecb0 0x7efe301f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 >> 172.21.15.72:0/2351560425 conn(0x7efe3009c450 msgr2=0x7efe3006bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 shutdown_connections 2023-12-12T09:52:38.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:38.795+0000 7efe38200640 1 -- 172.21.15.72:0/2351560425 wait complete. 2023-12-12T09:52:38.819 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":2070655694},{"type":"v1","addr":"172.21.15.72:6833","nonce":2070655694}]},"active_addr":"172.21.15.72:6833/2070655694","active_change":"2023-12-12T09:51:53.516111+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2974915640}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":3990966561}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":230861934}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":8182154}]}]} 2023-12-12T09:52:38.823 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-12T09:52:38.823 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-12T09:52:38.823 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:39.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.028+0000 7f526e6bb640 1 Processor -- start 2023-12-12T09:52:39.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.028+0000 7f526e6bb640 1 -- start start 2023-12-12T09:52:39.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f526e6bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268143e00 0x7f52680b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f526e6bb640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f526806c410 0x7f526815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f526e6bb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52680b3bb0 con 0x7f5268072180 2023-12-12T09:52:39.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f526e6bb640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52681441e0 con 0x7f526806c410 2023-12-12T09:52:39.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f526e6bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5268072680 con 0x7f5268143e00 2023-12-12T09:52:39.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f526cc58640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f526806c410 0x7f526815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f526cc58640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f526806c410 0x7f526815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40252/0 (socket says 172.21.15.72:40252) 2023-12-12T09:52:39.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f526cc58640 1 -- 172.21.15.72:0/807779007 learned_addr learned my addr 172.21.15.72:0/807779007 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:39.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.030+0000 7f5267fff640 1 --2- 172.21.15.72:0/807779007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268143e00 0x7f52680b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.031+0000 7f526cc58640 1 -- 172.21.15.72:0/807779007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268143e00 msgr2=0x7f52680b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:39.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.031+0000 7f526cc58640 1 --2- 172.21.15.72:0/807779007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268143e00 0x7f52680b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.031+0000 7f526cc58640 1 -- 172.21.15.72:0/807779007 >> v1:172.21.15.113:6789/0 conn(0x7f5268072180 legacy=0x7f52680b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.031+0000 7f526cc58640 1 -- 172.21.15.72:0/807779007 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52680b1e30 con 0x7f526806c410 2023-12-12T09:52:39.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.032+0000 7f526cc58640 1 --2- 172.21.15.72:0/807779007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526806c410 0x7f526815ecb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5260009e50 tx=0x7f526002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109cdb19117572eb server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:39.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.032+0000 7f5266ffd640 1 -- 172.21.15.72:0/807779007 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526002cb00 con 0x7f526806c410 2023-12-12T09:52:39.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.032+0000 7f5266ffd640 1 -- 172.21.15.72:0/807779007 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526002cc60 con 0x7f526806c410 2023-12-12T09:52:39.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.032+0000 7f5266ffd640 1 -- 172.21.15.72:0/807779007 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52600343f0 con 0x7f526806c410 2023-12-12T09:52:39.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.033+0000 7f526e6bb640 1 -- 172.21.15.72:0/807779007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526806c410 msgr2=0x7f526815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.033+0000 7f526e6bb640 1 --2- 172.21.15.72:0/807779007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526806c410 0x7f526815ecb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5260009e50 tx=0x7f526002bf60 comp rx=0 tx=0).stop 2023-12-12T09:52:39.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.033+0000 7f526e6bb640 1 -- 172.21.15.72:0/807779007 shutdown_connections 2023-12-12T09:52:39.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.033+0000 7f526e6bb640 1 --2- 172.21.15.72:0/807779007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526806c410 0x7f526815ecb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.033+0000 7f526e6bb640 1 --2- 172.21.15.72:0/807779007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268143e00 0x7f52680b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.033+0000 7f526e6bb640 1 -- 172.21.15.72:0/807779007 >> 172.21.15.72:0/807779007 conn(0x7f52680b0a90 msgr2=0x7f52680b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:39.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.033+0000 7f526e6bb640 1 -- 172.21.15.72:0/807779007 shutdown_connections 2023-12-12T09:52:39.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.033+0000 7f526e6bb640 1 -- 172.21.15.72:0/807779007 wait complete. 2023-12-12T09:52:39.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.035+0000 7f526e6bb640 1 Processor -- start 2023-12-12T09:52:39.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.035+0000 7f526e6bb640 1 -- start start 2023-12-12T09:52:39.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526e6bb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526806c410 0x7f52681f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526e6bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268072180 0x7f52680aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526e6bb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5268143e00 0x7f52680ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526e6bb640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680a9570 con 0x7f5268143e00 2023-12-12T09:52:39.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526e6bb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526815f8b0 con 0x7f526806c410 2023-12-12T09:52:39.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526e6bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680a88f0 con 0x7f5268072180 2023-12-12T09:52:39.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526cc58640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5268143e00 0x7f52680ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526cc58640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5268143e00 0x7f52680ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40256/0 (socket says 172.21.15.72:40256) 2023-12-12T09:52:39.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f526cc58640 1 -- 172.21.15.72:0/2294784685 learned_addr learned my addr 172.21.15.72:0/2294784685 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:39.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f5267fff640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526806c410 0x7f52681f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.036+0000 7f52677fe640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268072180 0x7f52680aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f526cc58640 1 -- 172.21.15.72:0/2294784685 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268072180 msgr2=0x7f52680aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f526cc58640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268072180 0x7f52680aa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f526cc58640 1 -- 172.21.15.72:0/2294784685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526806c410 msgr2=0x7f52681f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f526cc58640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526806c410 0x7f52681f0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f526cc58640 1 -- 172.21.15.72:0/2294784685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5260009b30 con 0x7f5268143e00 2023-12-12T09:52:39.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f526cc58640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5268143e00 0x7f52680ad910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5260009e50 tx=0x7f5260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:39.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f5264ff9640 1 -- 172.21.15.72:0/2294784685 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260009510 con 0x7f5268143e00 2023-12-12T09:52:39.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f5264ff9640 1 -- 172.21.15.72:0/2294784685 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526003e450 con 0x7f5268143e00 2023-12-12T09:52:39.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52680a8bd0 con 0x7f5268143e00 2023-12-12T09:52:39.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f5264ff9640 1 -- 172.21.15.72:0/2294784685 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526003e800 con 0x7f5268143e00 2023-12-12T09:52:39.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.037+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52680a9190 con 0x7f5268143e00 2023-12-12T09:52:39.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.039+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52680a96b0 con 0x7f5268143e00 2023-12-12T09:52:39.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.040+0000 7f5264ff9640 1 -- 172.21.15.72:0/2294784685 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f526003e960 con 0x7f5268143e00 2023-12-12T09:52:39.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.040+0000 7f5264ff9640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f524806ae40 0x7f524806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.041+0000 7f5264ff9640 1 -- 172.21.15.72:0/2294784685 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f52600ac610 con 0x7f5268143e00 2023-12-12T09:52:39.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.041+0000 7f5267fff640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f524806ae40 0x7f524806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.041+0000 7f5267fff640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f524806ae40 0x7f524806d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5258006fd0 tx=0x7f5258008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:39.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.045+0000 7f5264ff9640 1 -- 172.21.15.72:0/2294784685 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52680a96b0 con 0x7f5268143e00 2023-12-12T09:52:39.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.238+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52680a96b0 con 0x7f5268143e00 2023-12-12T09:52:39.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.240+0000 7f5264ff9640 1 -- 172.21.15.72:0/2294784685 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f52680a96b0 con 0x7f5268143e00 2023-12-12T09:52:39.242 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:39.242 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.242+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f524806ae40 msgr2=0x7f524806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.242+0000 7f526e6bb640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f524806ae40 0x7f524806d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5258006fd0 tx=0x7f5258008040 comp rx=0 tx=0).stop 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5268143e00 msgr2=0x7f52680ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5268143e00 0x7f52680ad910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5260009e50 tx=0x7f5260004a40 comp rx=0 tx=0).stop 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 shutdown_connections 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f524806ae40 0x7f524806d300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5268143e00 0x7f52680ad910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5268072180 0x7f52680aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 --2- 172.21.15.72:0/2294784685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526806c410 0x7f52681f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 >> 172.21.15.72:0/2294784685 conn(0x7f52680b0a90 msgr2=0x7f52680cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 shutdown_connections 2023-12-12T09:52:39.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.243+0000 7f526e6bb640 1 -- 172.21.15.72:0/2294784685 wait complete. 2023-12-12T09:52:39.263 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-12T09:52:39.264 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:39.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.471+0000 7f4dad161640 1 Processor -- start 2023-12-12T09:52:39.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.472+0000 7f4dad161640 1 -- start start 2023-12-12T09:52:39.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.473+0000 7f4dad161640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da806c690 0x7f4da80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.473+0000 7f4dad161640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80b07f0 con 0x7f4da80d0360 2023-12-12T09:52:39.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.473+0000 7f4dad161640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80b0a40 con 0x7f4da815d580 2023-12-12T09:52:39.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.473+0000 7f4dad161640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da815d960 con 0x7f4da806c690 2023-12-12T09:52:39.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.473+0000 7f4da659c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4da80d0360 0x7f4da806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52376/0 (socket says 172.21.15.72:52376) 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.473+0000 7f4da659c640 1 -- 172.21.15.72:0/3621435978 learned_addr learned my addr 172.21.15.72:0/3621435978 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da759e640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da806c690 0x7f4da80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16766193 0 0) 0x7f4da80b07f0 con 0x7f4da80d0360 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d94003520 con 0x7f4da80d0360 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2817887225 0 0) 0x7f4d94003520 con 0x7f4da80d0360 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da806c690 msgr2=0x7f4da80adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da806c690 0x7f4da80adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 >> v1:172.21.15.113:6789/0 conn(0x7f4da815d580 legacy=0x7f4da80cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80b0d20 con 0x7f4da80d0360 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f4d90001670 con 0x7f4da80d0360 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 >> v1:172.21.15.72:6789/0 conn(0x7f4da80d0360 legacy=0x7f4da806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da806c690 0x7f4d940039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d94003f70 0x7f4d940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94004d00 0x7f4d94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da815d960 con 0x7f4d94003f70 2023-12-12T09:52:39.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9400bac0 con 0x7f4d94004d00 2023-12-12T09:52:39.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.474+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9400bc80 con 0x7f4da806c690 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da659c640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d94003f70 0x7f4d940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da6d9d640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da806c690 0x7f4d940039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da759e640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94004d00 0x7f4d94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da659c640 1 -- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da806c690 msgr2=0x7f4d940039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da659c640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da806c690 0x7f4d940039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da659c640 1 -- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94004d00 msgr2=0x7f4d94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da659c640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94004d00 0x7f4d94005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da659c640 1 -- 172.21.15.72:0/3621435978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d94003520 con 0x7f4d94003f70 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.475+0000 7f4da759e640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94004d00 0x7f4d94005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.476+0000 7f4da659c640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d94003f70 0x7f4d940047c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4da8071aa0 tx=0x7f4d940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413ab15163b4fdeb server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.476+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90005ad0 con 0x7f4d94003f70 2023-12-12T09:52:39.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.476+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90005c30 con 0x7f4d94003f70 2023-12-12T09:52:39.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.476+0000 7f4da5d9b640 1 -- 172.21.15.72:0/3621435978 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d900072b0 con 0x7f4d94003f70 2023-12-12T09:52:39.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.477+0000 7f4dad161640 1 -- 172.21.15.72:0/3621435978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d94003f70 msgr2=0x7f4d940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.477+0000 7f4dad161640 1 --2- 172.21.15.72:0/3621435978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d94003f70 0x7f4d940047c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4da8071aa0 tx=0x7f4d940057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.477+0000 7f4da759e640 1 -- 172.21.15.72:0/3621435978 reap_dead start 2023-12-12T09:52:39.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.477+0000 7f4dad161640 1 -- 172.21.15.72:0/3621435978 shutdown_connections 2023-12-12T09:52:39.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.477+0000 7f4dad161640 1 -- 172.21.15.72:0/3621435978 >> 172.21.15.72:0/3621435978 conn(0x7f4da80d0e90 msgr2=0x7f4da80d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:39.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.477+0000 7f4dad161640 1 -- 172.21.15.72:0/3621435978 shutdown_connections 2023-12-12T09:52:39.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.477+0000 7f4dad161640 1 -- 172.21.15.72:0/3621435978 wait complete. 2023-12-12T09:52:39.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.479+0000 7f4dad161640 1 Processor -- start 2023-12-12T09:52:39.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.479+0000 7f4dad161640 1 -- start start 2023-12-12T09:52:39.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4dad161640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94003f70 0x7f4da80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4dad161640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4da80c53d0 0x7f4da8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4dad161640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da81563f0 0x7f4da815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4dad161640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80c5860 con 0x7f4da80c53d0 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4dad161640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80c5b40 con 0x7f4d94003f70 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4dad161640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80c5e20 con 0x7f4da81563f0 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4da659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4da80c53d0 0x7f4da8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4da659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4da80c53d0 0x7f4da8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40268/0 (socket says 172.21.15.72:40268) 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.480+0000 7f4da659c640 1 -- 172.21.15.72:0/3503180260 learned_addr learned my addr 172.21.15.72:0/3503180260 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da759e640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da81563f0 0x7f4da815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da6d9d640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94003f70 0x7f4da80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da659c640 1 -- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da81563f0 msgr2=0x7f4da815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da659c640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da81563f0 0x7f4da815b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da659c640 1 -- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94003f70 msgr2=0x7f4da80c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da659c640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94003f70 0x7f4da80c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da659c640 1 -- 172.21.15.72:0/3503180260 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80c6100 con 0x7f4da80c53d0 2023-12-12T09:52:39.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da759e640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da81563f0 0x7f4da815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:39.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da6d9d640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94003f70 0x7f4da80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:39.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4da659c640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4da80c53d0 0x7f4da8158ff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000c110 tx=0x7f4d90001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:39.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4d9f7fe640 1 -- 172.21.15.72:0/3503180260 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9003c040 con 0x7f4da80c53d0 2023-12-12T09:52:39.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4d9f7fe640 1 -- 172.21.15.72:0/3503180260 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90007500 con 0x7f4da80c53d0 2023-12-12T09:52:39.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4d9f7fe640 1 -- 172.21.15.72:0/3503180260 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9002c700 con 0x7f4da80c53d0 2023-12-12T09:52:39.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da80c63e0 con 0x7f4da80c53d0 2023-12-12T09:52:39.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.481+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da81568a0 con 0x7f4da80c53d0 2023-12-12T09:52:39.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.482+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da80ce8a0 con 0x7f4da80c53d0 2023-12-12T09:52:39.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.483+0000 7f4d9f7fe640 1 -- 172.21.15.72:0/3503180260 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d90032070 con 0x7f4da80c53d0 2023-12-12T09:52:39.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.484+0000 7f4d9f7fe640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d8806ae40 0x7f4d8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.484+0000 7f4d9f7fe640 1 -- 172.21.15.72:0/3503180260 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4d900aaec0 con 0x7f4da80c53d0 2023-12-12T09:52:39.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.485+0000 7f4da6d9d640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d8806ae40 0x7f4d8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.485+0000 7f4da6d9d640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d8806ae40 0x7f4d8806d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4d98007fd0 tx=0x7f4d98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:39.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.489+0000 7f4d9f7fe640 1 -- 172.21.15.72:0/3503180260 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4da80ce8a0 con 0x7f4da80c53d0 2023-12-12T09:52:39.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.670+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4da80ce8a0 con 0x7f4da80c53d0 2023-12-12T09:52:39.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.673+0000 7f4d9f7fe640 1 -- 172.21.15.72:0/3503180260 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f4da80ce8a0 con 0x7f4da80c53d0 2023-12-12T09:52:39.676 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:39.676 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:39.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d8806ae40 msgr2=0x7f4d8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d8806ae40 0x7f4d8806d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4d98007fd0 tx=0x7f4d98009040 comp rx=0 tx=0).stop 2023-12-12T09:52:39.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4da80c53d0 msgr2=0x7f4da8158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4da80c53d0 0x7f4da8158ff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000c110 tx=0x7f4d90001930 comp rx=0 tx=0).stop 2023-12-12T09:52:39.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 shutdown_connections 2023-12-12T09:52:39.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d8806ae40 0x7f4d8806d300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da81563f0 0x7f4da815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4da80c53d0 0x7f4da8158ff0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 --2- 172.21.15.72:0/3503180260 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d94003f70 0x7f4da80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 >> 172.21.15.72:0/3503180260 conn(0x7f4da80d0e90 msgr2=0x7f4da80d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:39.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 shutdown_connections 2023-12-12T09:52:39.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.676+0000 7f4dad161640 1 -- 172.21.15.72:0/3503180260 wait complete. 2023-12-12T09:52:39.695 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-12T09:52:39.695 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-12T09:52:39.696 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-12T09:52:39.696 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-12T09:52:39.696 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-12T09:52:39.697 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-12T09:52:39.697 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-12T09:52:39.698 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-12T09:52:39.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.986+0000 7fcc59175640 1 Processor -- start 2023-12-12T09:52:39.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.986+0000 7fcc59175640 1 -- start start 2023-12-12T09:52:39.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.988+0000 7fcc59175640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc540b3b00 0x7fcc5415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:39.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.988+0000 7fcc59175640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5415dc20 con 0x7fcc540b2700 2023-12-12T09:52:39.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.988+0000 7fcc59175640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc540b2270 con 0x7fcc54158fc0 2023-12-12T09:52:39.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.988+0000 7fcc59175640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5415d380 con 0x7fcc540b3b00 2023-12-12T09:52:39.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.990+0000 7fcc58974640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fcc540b2700 0x7fcc540cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52390/0 (socket says 172.21.15.72:52390) 2023-12-12T09:52:39.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.990+0000 7fcc58974640 1 -- 172.21.15.72:0/246434299 learned_addr learned my addr 172.21.15.72:0/246434299 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:39.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.990+0000 7fcc537fe640 1 --2- 172.21.15.72:0/246434299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc540b3b00 0x7fcc5415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:39.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.991+0000 7fcc537fe640 1 -- 172.21.15.72:0/246434299 >> v1:172.21.15.113:6789/0 conn(0x7fcc54158fc0 legacy=0x7fcc540abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:39.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.991+0000 7fcc537fe640 1 -- 172.21.15.72:0/246434299 >> v1:172.21.15.72:6789/0 conn(0x7fcc540b2700 legacy=0x7fcc540cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.991+0000 7fcc537fe640 1 -- 172.21.15.72:0/246434299 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5415d660 con 0x7fcc540b3b00 2023-12-12T09:52:39.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.991+0000 7fcc52ffd640 1 -- 172.21.15.72:0/246434299 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773951259 0 0) 0x7fcc5415dc20 con 0x7fcc540b2700 2023-12-12T09:52:39.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.993+0000 7fcc537fe640 1 --2- 172.21.15.72:0/246434299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc540b3b00 0x7fcc5415cc80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcc480097d0 tx=0x7fcc4802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48f3118978714b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:39.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.995+0000 7fcc52ffd640 1 -- 172.21.15.72:0/246434299 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48031780 con 0x7fcc540b3b00 2023-12-12T09:52:39.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.995+0000 7fcc52ffd640 1 -- 172.21.15.72:0/246434299 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc480318e0 con 0x7fcc540b3b00 2023-12-12T09:52:39.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.995+0000 7fcc52ffd640 1 -- 172.21.15.72:0/246434299 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48031c70 con 0x7fcc540b3b00 2023-12-12T09:52:39.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.996+0000 7fcc59175640 1 -- 172.21.15.72:0/246434299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc540b3b00 msgr2=0x7fcc5415cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:39.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.996+0000 7fcc59175640 1 --2- 172.21.15.72:0/246434299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc540b3b00 0x7fcc5415cc80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcc480097d0 tx=0x7fcc4802b820 comp rx=0 tx=0).stop 2023-12-12T09:52:39.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.997+0000 7fcc59175640 1 -- 172.21.15.72:0/246434299 shutdown_connections 2023-12-12T09:52:39.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.997+0000 7fcc59175640 1 --2- 172.21.15.72:0/246434299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc540b3b00 0x7fcc5415cc80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:39.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.997+0000 7f9ef2b35640 1 Processor -- start 2023-12-12T09:52:39.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.997+0000 7fcc59175640 1 -- 172.21.15.72:0/246434299 >> 172.21.15.72:0/246434299 conn(0x7fcc5406a460 msgr2=0x7fcc5406a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:39.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.997+0000 7fcc59175640 1 -- 172.21.15.72:0/246434299 shutdown_connections 2023-12-12T09:52:40.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.998+0000 7fcc59175640 1 -- 172.21.15.72:0/246434299 wait complete. 2023-12-12T09:52:40.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.997+0000 7f9ef2b35640 1 -- start start 2023-12-12T09:52:40.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.999+0000 7f9ef2b35640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9ee4005e60 0x7f9ee40062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.999+0000 7f9ef2b35640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee4054300 con 0x7f9ee40067e0 2023-12-12T09:52:40.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.999+0000 7f9ef2b35640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee40536f0 con 0x7f9ee40eacd0 2023-12-12T09:52:40.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:39.999+0000 7f9ef2b35640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee40539d0 con 0x7f9ee4005e60 2023-12-12T09:52:40.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.001+0000 7f9ef1332640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9ee4005e60 0x7f9ee40062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.001+0000 7fcc59175640 1 Processor -- start 2023-12-12T09:52:40.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.001+0000 7f9ef2334640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f9ee40067e0 0x7f9ee4006c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52394/0 (socket says 172.21.15.72:52394) 2023-12-12T09:52:40.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.001+0000 7f9ef2334640 1 -- 172.21.15.72:0/2207025168 learned_addr learned my addr 172.21.15.72:0/2207025168 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.002+0000 7fcc59175640 1 -- start start 2023-12-12T09:52:40.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7fcc59175640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc540b2700 0x7fcc540cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7fcc59175640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc540b3b00 0x7fcc540999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7fcc59175640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc54158fc0 0x7fcc541536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7fcc59175640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc541eea90 con 0x7fcc540b2700 2023-12-12T09:52:40.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7fcc59175640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc541f24e0 con 0x7fcc540b3b00 2023-12-12T09:52:40.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7fcc59175640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc541f27c0 con 0x7fcc54158fc0 2023-12-12T09:52:40.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7f81c1789640 1 Processor -- start 2023-12-12T09:52:40.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7f81c1789640 1 -- start start 2023-12-12T09:52:40.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.005+0000 7fcc53fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc540b2700 0x7fcc540cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.005+0000 7fcc53fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc540b2700 0x7fcc540cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40286/0 (socket says 172.21.15.72:40286) 2023-12-12T09:52:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.005+0000 7f81c1789640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f81bc06d520 0x7f81bc0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.005+0000 7f81c1789640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc0cde40 con 0x7f81bc0708d0 2023-12-12T09:52:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.005+0000 7f81c1789640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc0cd230 con 0x7f81bc15f260 2023-12-12T09:52:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.005+0000 7f81c1789640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0cd040 con 0x7f81bc06d520 2023-12-12T09:52:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.005+0000 7fcc58974640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc54158fc0 0x7fcc541536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.006+0000 7fcc58974640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc54158fc0 0x7fcc541536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49346/0 (socket says 172.21.15.72:49346) 2023-12-12T09:52:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.006+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc540b3b00 0x7fcc540999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.006+0000 7fcc58974640 1 -- 172.21.15.72:0/825903993 learned_addr learned my addr 172.21.15.72:0/825903993 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7f9ef0b31640 1 -- 172.21.15.72:0/2207025168 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552863003 0 0) 0x7f9ee40536f0 con 0x7f9ee40eacd0 2023-12-12T09:52:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.003+0000 7f9ef0b31640 1 -- 172.21.15.72:0/2207025168 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed8003520 con 0x7f9ee40eacd0 2023-12-12T09:52:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.004+0000 7f9ef1332640 1 -- 172.21.15.72:0/2207025168 >> v1:172.21.15.113:6789/0 conn(0x7f9ee40eacd0 legacy=0x7f9ee404fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.004+0000 7f9ef1332640 1 -- 172.21.15.72:0/2207025168 >> v1:172.21.15.72:6789/0 conn(0x7f9ee40067e0 legacy=0x7f9ee4006c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.004+0000 7f9ef1332640 1 -- 172.21.15.72:0/2207025168 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4053cb0 con 0x7f9ee4005e60 2023-12-12T09:52:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.006+0000 7f9ef1332640 1 --2- 172.21.15.72:0/2207025168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 0x7f9ee40062a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80097a0 tx=0x7f9ee802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21da16126280cee server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.006+0000 7f81ba7fc640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f81bc0708d0 0x7f81bc1691b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52404/0 (socket says 172.21.15.72:52404) 2023-12-12T09:52:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.006+0000 7f81ba7fc640 1 -- 172.21.15.72:0/522258845 learned_addr learned my addr 172.21.15.72:0/522258845 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.006+0000 7f81baffd640 1 --1- 172.21.15.72:0/522258845 >> v1:172.21.15.113:6789/0 conn(0x7f81bc15f260 0x7f81bc0701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54868/0 (socket says 172.21.15.72:54868) 2023-12-12T09:52:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.007+0000 7f81bb7fe640 1 --2- 172.21.15.72:0/522258845 >> v2:172.21.15.72:3300/0 conn(0x7f81bc06d520 0x7f81bc0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.007+0000 7fcc53fff640 1 -- 172.21.15.72:0/825903993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc54158fc0 msgr2=0x7fcc541536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.007+0000 7fcc53fff640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc54158fc0 0x7fcc541536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.007+0000 7fcc53fff640 1 -- 172.21.15.72:0/825903993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc540b3b00 msgr2=0x7fcc540999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.007+0000 7f9ef0b31640 1 -- 172.21.15.72:0/2207025168 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8030610 con 0x7f9ee4005e60 2023-12-12T09:52:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.007+0000 7f9ef0b31640 1 -- 172.21.15.72:0/2207025168 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee8030770 con 0x7f9ee4005e60 2023-12-12T09:52:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.007+0000 7f9ef0b31640 1 -- 172.21.15.72:0/2207025168 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8030b00 con 0x7f9ee4005e60 2023-12-12T09:52:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.007+0000 7f81b9ffb640 1 -- 172.21.15.72:0/522258845 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807791673 0 0) 0x7f81bc0cd230 con 0x7f81bc15f260 2023-12-12T09:52:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.008+0000 7f81b9ffb640 1 -- 172.21.15.72:0/522258845 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a4003520 con 0x7f81bc15f260 2023-12-12T09:52:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.008+0000 7fcc53fff640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc540b3b00 0x7fcc540999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.008+0000 7f81bb7fe640 1 -- 172.21.15.72:0/522258845 >> v1:172.21.15.113:6789/0 conn(0x7f81bc15f260 legacy=0x7f81bc0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.008+0000 7f81bb7fe640 1 -- 172.21.15.72:0/522258845 >> v1:172.21.15.72:6789/0 conn(0x7f81bc0708d0 legacy=0x7f81bc1691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.008+0000 7f81bb7fe640 1 -- 172.21.15.72:0/522258845 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc0cd510 con 0x7f81bc06d520 2023-12-12T09:52:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.008+0000 7fcc53fff640 1 -- 172.21.15.72:0/825903993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc48009400 con 0x7fcc540b2700 2023-12-12T09:52:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.008+0000 7fcc53fff640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc540b2700 0x7fcc540cae70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400bf90 tx=0x7fcc44009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7f9ef2b35640 1 -- 172.21.15.72:0/2207025168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 msgr2=0x7f9ee40062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7f9ef2b35640 1 --2- 172.21.15.72:0/2207025168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 0x7f9ee40062a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80097a0 tx=0x7f9ee802b960 comp rx=0 tx=0).stop 2023-12-12T09:52:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7fcc537fe640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc540b3b00 0x7fcc540999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7fcc58974640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc54158fc0 0x7fcc541536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7f9ef2b35640 1 -- 172.21.15.72:0/2207025168 shutdown_connections 2023-12-12T09:52:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7f9ef2b35640 1 --2- 172.21.15.72:0/2207025168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 0x7f9ee40062a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7f9ef2b35640 1 -- 172.21.15.72:0/2207025168 >> 172.21.15.72:0/2207025168 conn(0x7f9ee4009f80 msgr2=0x7f9ee400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44014440 con 0x7fcc540b2700 2023-12-12T09:52:40.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc440145a0 con 0x7fcc540b2700 2023-12-12T09:52:40.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.009+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44014930 con 0x7fcc540b2700 2023-12-12T09:52:40.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.010+0000 7f81bb7fe640 1 --2- 172.21.15.72:0/522258845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc06d520 0x7f81bc0ccb00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009d90 tx=0x7f81ac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e9ba231dd2695d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.010+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc540ac5b0 con 0x7fcc540b2700 2023-12-12T09:52:40.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.010+0000 7f81b9ffb640 1 -- 172.21.15.72:0/522258845 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac02cd10 con 0x7f81bc06d520 2023-12-12T09:52:40.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.010+0000 7f81b9ffb640 1 -- 172.21.15.72:0/522258845 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac02ce70 con 0x7f81bc06d520 2023-12-12T09:52:40.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.010+0000 7f81b9ffb640 1 -- 172.21.15.72:0/522258845 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac02cd10 con 0x7f81bc06d520 2023-12-12T09:52:40.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.011+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc541f2d80 con 0x7fcc540b2700 2023-12-12T09:52:40.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.012+0000 7ff5e3e7c640 1 Processor -- start 2023-12-12T09:52:40.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.012+0000 7f81c1789640 1 -- 172.21.15.72:0/522258845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc06d520 msgr2=0x7f81bc0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.012+0000 7f81c1789640 1 --2- 172.21.15.72:0/522258845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc06d520 0x7f81bc0ccb00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009d90 tx=0x7f81ac02bf60 comp rx=0 tx=0).stop 2023-12-12T09:52:40.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.011+0000 7f9ef2b35640 1 -- 172.21.15.72:0/2207025168 shutdown_connections 2023-12-12T09:52:40.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.011+0000 7f9ef2b35640 1 -- 172.21.15.72:0/2207025168 wait complete. 2023-12-12T09:52:40.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.013+0000 7f9ef2b35640 1 Processor -- start 2023-12-12T09:52:40.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.012+0000 7f81c1789640 1 -- 172.21.15.72:0/522258845 shutdown_connections 2023-12-12T09:52:40.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.012+0000 7f81c1789640 1 --2- 172.21.15.72:0/522258845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc06d520 0x7f81bc0ccb00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.012+0000 7f81c1789640 1 -- 172.21.15.72:0/522258845 >> 172.21.15.72:0/522258845 conn(0x7f81bc0b0a90 msgr2=0x7f81bc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.013+0000 7f81c1789640 1 -- 172.21.15.72:0/522258845 shutdown_connections 2023-12-12T09:52:40.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.012+0000 7ff5e3e7c640 1 -- start start 2023-12-12T09:52:40.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.014+0000 7fcc3a7fc640 1 -- 172.21.15.72:0/825903993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc540ac430 con 0x7fcc540b2700 2023-12-12T09:52:40.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.014+0000 7ff5e3e7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc0b1580 0x7ff5dc070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.014+0000 7ff5e3e7c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff5dc06cc90 0x7ff5dc0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.014+0000 7ff5e3e7c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5dc06d650 con 0x7ff5dc15a580 2023-12-12T09:52:40.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.014+0000 7ff5e3e7c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5dc0ccd70 con 0x7ff5dc06cc90 2023-12-12T09:52:40.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.014+0000 7ff5e3e7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0cd050 con 0x7ff5dc0b1580 2023-12-12T09:52:40.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.015+0000 7ff5e1417640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc0b1580 0x7ff5dc070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.014+0000 7f9ef2b35640 1 -- start start 2023-12-12T09:52:40.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.015+0000 7ff5e1417640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc0b1580 0x7ff5dc070a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49348/0 (socket says 172.21.15.72:49348) 2023-12-12T09:52:40.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.015+0000 7ff5e1417640 1 -- 172.21.15.72:0/1312952951 learned_addr learned my addr 172.21.15.72:0/1312952951 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef2b35640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 0x7f9ee402b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.013+0000 7f81c1789640 1 -- 172.21.15.72:0/522258845 wait complete. 2023-12-12T09:52:40.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f81c1789640 1 Processor -- start 2023-12-12T09:52:40.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef2b35640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee40067e0 0x7f9ee4036810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f81c1789640 1 -- start start 2023-12-12T09:52:40.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef2b35640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee40eacd0 0x7f9ee4179320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef1b33640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 0x7f9ee402b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7ff5e2419640 1 --2- 172.21.15.72:0/1312952951 >> v2:172.21.15.72:3300/0 conn(0x7ff5dc06cc90 0x7ff5dc0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef2b35640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee402b980 con 0x7f9ee4005e60 2023-12-12T09:52:40.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef1b33640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 0x7f9ee402b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40328/0 (socket says 172.21.15.72:40328) 2023-12-12T09:52:40.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7ff5e1417640 1 -- 172.21.15.72:0/1312952951 >> v2:172.21.15.72:3300/0 conn(0x7ff5dc06cc90 msgr2=0x7ff5dc0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef2b35640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40d6fe0 con 0x7f9ee40067e0 2023-12-12T09:52:40.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef2b35640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40d72c0 con 0x7f9ee40eacd0 2023-12-12T09:52:40.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7f9ef1b33640 1 -- 172.21.15.72:0/1295220659 learned_addr learned my addr 172.21.15.72:0/1295220659 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.017+0000 7f9ef2334640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee40eacd0 0x7f9ee4179320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.017+0000 7f9ef1332640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee40067e0 0x7f9ee4036810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f81c1789640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81bc06d520 0x7f81bc15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc4401d3b0 con 0x7fcc540b2700 2023-12-12T09:52:40.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f81c1789640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc0708d0 0x7f81bc1ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f9ef1332640 1 -- 172.21.15.72:0/1295220659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee40eacd0 msgr2=0x7f9ee4179320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f9ef1332640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee40eacd0 0x7f9ee4179320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f9ef1332640 1 -- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 msgr2=0x7f9ee402b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.016+0000 7ff5e1417640 1 --2- 172.21.15.72:0/1312952951 >> v2:172.21.15.72:3300/0 conn(0x7ff5dc06cc90 0x7ff5dc0b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.017+0000 7ff5e1417640 1 -- 172.21.15.72:0/1312952951 >> v1:172.21.15.72:6789/0 conn(0x7ff5dc15a580 legacy=0x7ff5dc153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.017+0000 7ff5e1417640 1 -- 172.21.15.72:0/1312952951 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0cd330 con 0x7ff5dc0b1580 2023-12-12T09:52:40.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f9ef1332640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 0x7f9ee402b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7f9ef1332640 1 -- 172.21.15.72:0/1295220659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee8009400 con 0x7f9ee40067e0 2023-12-12T09:52:40.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7f9ef2334640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee40eacd0 0x7f9ee4179320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:40.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7f9ef1b33640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ee4005e60 0x7f9ee402b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:40.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f81c1789640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81bc15f260 0x7f81bc15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f81c1789640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc200210 con 0x7f81bc0708d0 2023-12-12T09:52:40.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f81c1789640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc2004f0 con 0x7f81bc15f260 2023-12-12T09:52:40.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.018+0000 7f81c1789640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc2007d0 con 0x7f81bc06d520 2023-12-12T09:52:40.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7f81ba7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc0708d0 0x7f81bc1ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7f81ba7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc0708d0 0x7f81bc1ff9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40330/0 (socket says 172.21.15.72:40330) 2023-12-12T09:52:40.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7f81bb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81bc15f260 0x7f81bc15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7f81ba7fc640 1 -- 172.21.15.72:0/3730282001 learned_addr learned my addr 172.21.15.72:0/3730282001 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7ff5e1417640 1 --2- 172.21.15.72:0/1312952951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc0b1580 0x7ff5dc070a80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009870 tx=0x7ff5cc02ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=684522490488f74d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.019+0000 7f81baffd640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81bc06d520 0x7f81bc15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7f81ba7fc640 1 -- 172.21.15.72:0/3730282001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81bc06d520 msgr2=0x7f81bc15ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7f81ba7fc640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81bc06d520 0x7f81bc15ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7f81ba7fc640 1 -- 172.21.15.72:0/3730282001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81bc15f260 msgr2=0x7f81bc15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7f81ba7fc640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81bc15f260 0x7f81bc15cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7f81ba7fc640 1 -- 172.21.15.72:0/3730282001 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac009a70 con 0x7f81bc0708d0 2023-12-12T09:52:40.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7f9ef1332640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee40067e0 0x7f9ee4036810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8033000 tx=0x7f9ee8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7ff5e0c16640 1 -- 172.21.15.72:0/1312952951 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc02f850 con 0x7ff5dc0b1580 2023-12-12T09:52:40.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7f81ba7fc640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc0708d0 0x7f81bc1ff9b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c4a0 tx=0x7f81b400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.020+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee803e040 con 0x7f9ee40067e0 2023-12-12T09:52:40.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.021+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee8039c10 con 0x7f9ee40067e0 2023-12-12T09:52:40.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.021+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee803e040 con 0x7f9ee40067e0 2023-12-12T09:52:40.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.021+0000 7ff5e0c16640 1 -- 172.21.15.72:0/1312952951 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc02f9b0 con 0x7ff5dc0b1580 2023-12-12T09:52:40.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.021+0000 7f81a37fe640 1 -- 172.21.15.72:0/3730282001 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4005ce0 con 0x7f81bc0708d0 2023-12-12T09:52:40.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.021+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee40d75a0 con 0x7f9ee40067e0 2023-12-12T09:52:40.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.021+0000 7ff5e0c16640 1 -- 172.21.15.72:0/1312952951 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc02f850 con 0x7ff5dc0b1580 2023-12-12T09:52:40.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.021+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee40d7b60 con 0x7f9ee40067e0 2023-12-12T09:52:40.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.022+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc200ab0 con 0x7f81bc0708d0 2023-12-12T09:52:40.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.022+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc1ffef0 con 0x7f81bc0708d0 2023-12-12T09:52:40.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.022+0000 7f81a37fe640 1 -- 172.21.15.72:0/3730282001 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b4005e40 con 0x7f81bc0708d0 2023-12-12T09:52:40.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.022+0000 7f81a37fe640 1 -- 172.21.15.72:0/3730282001 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4007270 con 0x7f81bc0708d0 2023-12-12T09:52:40.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.023+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/1312952951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc0b1580 msgr2=0x7ff5dc070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.023+0000 7fcc50ff9640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc4006af10 0x7fcc4006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.023+0000 7ff5e3e7c640 1 --2- 172.21.15.72:0/1312952951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc0b1580 0x7ff5dc070a80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009870 tx=0x7ff5cc02ba30 comp rx=0 tx=0).stop 2023-12-12T09:52:40.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.023+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/1312952951 shutdown_connections 2023-12-12T09:52:40.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.024+0000 7ff5e3e7c640 1 --2- 172.21.15.72:0/1312952951 >> v2:172.21.15.72:3300/0 conn(0x7ff5dc06cc90 0x7ff5dc0b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.023+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcc44089c50 con 0x7fcc540b2700 2023-12-12T09:52:40.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.024+0000 7ff5e3e7c640 1 --2- 172.21.15.72:0/1312952951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc0b1580 0x7ff5dc070a80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.024+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/1312952951 >> 172.21.15.72:0/1312952951 conn(0x7ff5dc0b0a90 msgr2=0x7ff5dc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.025+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ee404eee0 con 0x7f9ee40067e0 2023-12-12T09:52:40.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/1312952951 shutdown_connections 2023-12-12T09:52:40.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7f8ac6490640 1 Processor -- start 2023-12-12T09:52:40.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.023+0000 7fde4dd78640 1 Processor -- start 2023-12-12T09:52:40.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.023+0000 7fde4dd78640 1 -- start start 2023-12-12T09:52:40.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7fde4dd78640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde480708d0 0x7fde480ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7fde4dd78640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 0x7fde481626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7fde4dd78640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde480cde40 con 0x7fde4815f260 2023-12-12T09:52:40.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7fde4dd78640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde480cd040 con 0x7fde480708d0 2023-12-12T09:52:40.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7fde4dd78640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde48162bf0 con 0x7fde4806d520 2023-12-12T09:52:40.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/1312952951 wait complete. 2023-12-12T09:52:40.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7fcc50ff9640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7fcc40074550 0x7fcc40076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7fde46ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde480708d0 0x7fde480ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7fde477fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fde4815f260 0x7fde480701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52416/0 (socket says 172.21.15.72:52416) 2023-12-12T09:52:40.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7fde477fe640 1 -- 172.21.15.72:0/1354874180 learned_addr learned my addr 172.21.15.72:0/1354874180 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.026+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ee802f370 con 0x7f9ee40067e0 2023-12-12T09:52:40.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7f9ee27fc640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ed006aea0 0x7f9ed006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ee80ab6a0 con 0x7f9ee40067e0 2023-12-12T09:52:40.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.028+0000 7f9ee27fc640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f9ed00743e0 0x7f9ed0076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.028+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 --> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ed0076ef0 con 0x7f9ed00743e0 2023-12-12T09:52:40.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7f8ac6490640 1 -- start start 2023-12-12T09:52:40.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f8ac6490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac015a580 0x7f8ac006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7fde47fff640 1 --2- 172.21.15.72:0/1354874180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 0x7fde481626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.028+0000 7fde46ffd640 1 -- 172.21.15.72:0/1354874180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 msgr2=0x7fde481626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.028+0000 7fde46ffd640 1 --2- 172.21.15.72:0/1354874180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 0x7fde481626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.028+0000 7fde46ffd640 1 -- 172.21.15.72:0/1354874180 >> v1:172.21.15.72:6789/0 conn(0x7fde4815f260 legacy=0x7fde480701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.028+0000 7fde46ffd640 1 -- 172.21.15.72:0/1354874180 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480cd230 con 0x7fde480708d0 2023-12-12T09:52:40.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f8ac6490640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac0153ce0 con 0x7f8ac0153900 2023-12-12T09:52:40.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f8ac6490640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac006d650 con 0x7f8ac006d1d0 2023-12-12T09:52:40.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f8ac6490640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0071190 con 0x7f8ac015a580 2023-12-12T09:52:40.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.027+0000 7f81a37fe640 1 -- 172.21.15.72:0/3730282001 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81b4007490 con 0x7f81bc0708d0 2023-12-12T09:52:40.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f81a37fe640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f81a806ae40 0x7f81a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f81a37fe640 1 -- 172.21.15.72:0/3730282001 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f81b4088d80 con 0x7f81bc0708d0 2023-12-12T09:52:40.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f9ef1b33640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ed006aea0 0x7f9ed006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f8ac4a2d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8ac006d1d0 0x7f8ac00b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54874/0 (socket says 172.21.15.72:54874) 2023-12-12T09:52:40.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7f8ac4a2d640 1 -- 172.21.15.72:0/378707440 learned_addr learned my addr 172.21.15.72:0/378707440 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.030+0000 7f81baffd640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f81a806ae40 0x7f81a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.030+0000 7f9ef2334640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f9ed00743e0 0x7f9ed0076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.030+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 --> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc40077060 con 0x7fcc40074550 2023-12-12T09:52:40.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc4408c270 con 0x7fcc540b2700 2023-12-12T09:52:40.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7fcc537fe640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc4006af10 0x7fcc4006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f9ef1b33640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ed006aea0 0x7f9ed006d360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9edc001010 tx=0x7f9edc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.030+0000 7f8ab77fe640 1 --2- 172.21.15.72:0/378707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac015a580 0x7f8ac006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f8abf7fe640 1 -- 172.21.15.72:0/378707440 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801437918 0 0) 0x7f8ab8001110 con 0x7f8ac006d1d0 2023-12-12T09:52:40.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f8abf7fe640 1 -- 172.21.15.72:0/378707440 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aac007930 con 0x7f8ac006d1d0 2023-12-12T09:52:40.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f81baffd640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f81a806ae40 0x7f81a806d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81b00011e0 tx=0x7f81b0007b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7fcc58974640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7fcc40074550 0x7fcc40076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f81c1789640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7f819c0015e0 0x7f819c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 --> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f819c006b40 con 0x7f819c0015e0 2023-12-12T09:52:40.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f8ab77fe640 1 -- 172.21.15.72:0/378707440 >> v1:172.21.15.113:6789/0 conn(0x7f8ac006d1d0 legacy=0x7f8ac00b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f8ab77fe640 1 -- 172.21.15.72:0/378707440 >> v1:172.21.15.72:6789/0 conn(0x7f8ac0153900 legacy=0x7f8ac0070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7f8ab77fe640 1 -- 172.21.15.72:0/378707440 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00cca90 con 0x7f8ac015a580 2023-12-12T09:52:40.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.032+0000 7fcc58974640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7fcc40074550 0x7fcc40076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.032+0000 7f9ef2334640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f9ed00743e0 0x7f9ed0076820 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.033+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ee8076460 con 0x7f9ee40067e0 2023-12-12T09:52:40.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.033+0000 7fde46ffd640 1 --2- 172.21.15.72:0/1354874180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 0x7fde480ccb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097a0 tx=0x7fde3c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9860ec74a506a0ab server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.033+0000 7f81bb7fe640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7f819c0015e0 0x7f819c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.033+0000 7fde467fc640 1 -- 172.21.15.72:0/1354874180 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c02d040 con 0x7fde480708d0 2023-12-12T09:52:40.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.029+0000 7ff5e3e7c640 1 Processor -- start 2023-12-12T09:52:40.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.031+0000 7ff5e3e7c640 1 -- start start 2023-12-12T09:52:40.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.034+0000 7fde467fc640 1 -- 172.21.15.72:0/1354874180 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c031810 con 0x7fde480708d0 2023-12-12T09:52:40.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.034+0000 7fcc537fe640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc4006af10 0x7fcc4006d3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc48036000 tx=0x7fcc4803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.034+0000 7ff5e3e7c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5dc06cc90 0x7ff5dc16cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.034+0000 7f8ab77fe640 1 --2- 172.21.15.72:0/378707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac015a580 0x7f8ac006cc90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d8d0 tx=0x7f8aa802bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad981aeb041569 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.034+0000 7f81bb7fe640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7f819c0015e0 0x7f819c003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.034+0000 7ff5e1c18640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5dc06cc90 0x7ff5dc16cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e1c18640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5dc06cc90 0x7ff5dc16cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40340/0 (socket says 172.21.15.72:40340) 2023-12-12T09:52:40.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e1c18640 1 -- 172.21.15.72:0/2792713820 learned_addr learned my addr 172.21.15.72:0/2792713820 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e3e7c640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5dc0b1580 0x7ff5dc15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e3e7c640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc15a580 0x7ff5dc15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc16e870 con 0x7ff5dc06cc90 2023-12-12T09:52:40.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc140890 con 0x7ff5dc0b1580 2023-12-12T09:52:40.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc16c420 con 0x7ff5dc15a580 2023-12-12T09:52:40.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 <== osd.0 v2:172.21.15.72:6823/2303781576 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fcc40077060 con 0x7fcc40074550 2023-12-12T09:52:40.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 <== osd.1 v2:172.21.15.72:6800/1034360867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9ed0076ef0 con 0x7f9ed00743e0 2023-12-12T09:52:40.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e1c18640 1 -- 172.21.15.72:0/2792713820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc15a580 msgr2=0x7ff5dc15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e1c18640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5dc15a580 0x7ff5dc15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e1c18640 1 -- 172.21.15.72:0/2792713820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5dc0b1580 msgr2=0x7ff5dc15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e1c18640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5dc0b1580 0x7ff5dc15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7ff5e1c18640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc0094d0 con 0x7ff5dc06cc90 2023-12-12T09:52:40.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7f8abf7fe640 1 -- 172.21.15.72:0/378707440 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa802d8a0 con 0x7f8ac015a580 2023-12-12T09:52:40.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7f8abf7fe640 1 -- 172.21.15.72:0/378707440 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa802da00 con 0x7f8ac015a580 2023-12-12T09:52:40.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7f8abf7fe640 1 -- 172.21.15.72:0/378707440 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa802dd90 con 0x7f8ac015a580 2023-12-12T09:52:40.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7fde4dd78640 1 -- 172.21.15.72:0/1354874180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 msgr2=0x7fde480ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.035+0000 7fde467fc640 1 -- 172.21.15.72:0/1354874180 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c02d040 con 0x7fde480708d0 2023-12-12T09:52:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7ff5e1c18640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5dc06cc90 0x7ff5dc16cf00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c400 tx=0x7ff5d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7f81a37fe640 1 -- 172.21.15.72:0/3730282001 <== osd.2 v2:172.21.15.72:6816/323046314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f819c006b40 con 0x7f819c0015e0 2023-12-12T09:52:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80052d0 con 0x7ff5dc06cc90 2023-12-12T09:52:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7fde4dd78640 1 --2- 172.21.15.72:0/1354874180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 0x7fde480ccb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0097a0 tx=0x7fde3c02b8b0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.036+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc16c700 con 0x7ff5dc06cc90 2023-12-12T09:52:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.037+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc139df0 con 0x7ff5dc06cc90 2023-12-12T09:52:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.037+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d8005430 con 0x7ff5dc06cc90 2023-12-12T09:52:40.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.037+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8003830 con 0x7ff5dc06cc90 2023-12-12T09:52:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.038+0000 7fde4dd78640 1 -- 172.21.15.72:0/1354874180 shutdown_connections 2023-12-12T09:52:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.038+0000 7fde4dd78640 1 --2- 172.21.15.72:0/1354874180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 0x7fde481626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.038+0000 7fde4dd78640 1 --2- 172.21.15.72:0/1354874180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 0x7fde480ccb00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.038+0000 7fde4dd78640 1 -- 172.21.15.72:0/1354874180 >> 172.21.15.72:0/1354874180 conn(0x7fde480b0a90 msgr2=0x7fde480b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.038+0000 7fde4dd78640 1 -- 172.21.15.72:0/1354874180 shutdown_connections 2023-12-12T09:52:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.038+0000 7fde4dd78640 1 -- 172.21.15.72:0/1354874180 wait complete. 2023-12-12T09:52:40.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.038+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5c4000fc0 con 0x7ff5dc06cc90 2023-12-12T09:52:40.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.041+0000 7fde4dd78640 1 Processor -- start 2023-12-12T09:52:40.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.041+0000 7fde4dd78640 1 -- start start 2023-12-12T09:52:40.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.041+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5d8003990 con 0x7ff5dc06cc90 2023-12-12T09:52:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.037+0000 7f8ac6490640 1 -- 172.21.15.72:0/378707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac015a580 msgr2=0x7f8ac006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.037+0000 7f8ac6490640 1 --2- 172.21.15.72:0/378707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac015a580 0x7f8ac006cc90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d8d0 tx=0x7f8aa802bae0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.037+0000 7f8ac6490640 1 -- 172.21.15.72:0/378707440 shutdown_connections 2023-12-12T09:52:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.037+0000 7f8ac6490640 1 --2- 172.21.15.72:0/378707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac015a580 0x7f8ac006cc90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.037+0000 7f8ac6490640 1 -- 172.21.15.72:0/378707440 >> 172.21.15.72:0/378707440 conn(0x7f8ac00b0a90 msgr2=0x7f8ac00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.042+0000 7f8ac6490640 1 -- 172.21.15.72:0/378707440 shutdown_connections 2023-12-12T09:52:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.043+0000 7ff5d27fc640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5c006ae40 0x7ff5c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.043+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff5d8088c50 con 0x7ff5dc06cc90 2023-12-12T09:52:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.043+0000 7f8ac6490640 1 -- 172.21.15.72:0/378707440 wait complete. 2023-12-12T09:52:40.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.044+0000 7ff5e1417640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5c006ae40 0x7ff5c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.044+0000 7ff5d27fc640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7ff5c0074400 0x7ff5c0076840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.043+0000 7fde4dd78640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 0x7fde480b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.044+0000 7fde4dd78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 0x7fde4814aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7ff5e2419640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7ff5c0074400 0x7ff5c0076840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7ff5e1417640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5c006ae40 0x7ff5c006d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc036000 tx=0x7ff5cc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5c0076f10 con 0x7ff5c0074400 2023-12-12T09:52:40.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5d808b310 con 0x7ff5dc06cc90 2023-12-12T09:52:40.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7fde477fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 0x7fde480b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7fde4dd78640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde4815f260 0x7fde4814b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7fde477fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 0x7fde480b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49408/0 (socket says 172.21.15.72:49408) 2023-12-12T09:52:40.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7fde46ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 0x7fde4814aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7fde47fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde4815f260 0x7fde4814b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.045+0000 7fde46ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 0x7fde4814aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57744/0 (socket says 172.21.15.72:57744) 2023-12-12T09:52:40.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.046+0000 7fde47fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde4815f260 0x7fde4814b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40354/0 (socket says 172.21.15.72:40354) 2023-12-12T09:52:40.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.046+0000 7ff5e2419640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7ff5c0074400 0x7ff5c0076840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.046+0000 7f8ac6490640 1 Processor -- start 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.047+0000 7f8ac6490640 1 -- start start 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.046+0000 7fde4dd78640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48163230 con 0x7fde4815f260 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.046+0000 7fde4dd78640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4814b960 con 0x7fde480708d0 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.046+0000 7fde4dd78640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4814bbf0 con 0x7fde4806d520 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.046+0000 7fde477fe640 1 -- 172.21.15.72:0/1672888712 learned_addr learned my addr 172.21.15.72:0/1672888712 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.047+0000 7fde47fff640 1 -- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 msgr2=0x7fde480b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.047+0000 7fde47fff640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde4806d520 0x7fde480b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.047+0000 7fde47fff640 1 -- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 msgr2=0x7fde4814aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.047+0000 7fde47fff640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde480708d0 0x7fde4814aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.047+0000 7fde47fff640 1 -- 172.21.15.72:0/1672888712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c009400 con 0x7fde4815f260 2023-12-12T09:52:40.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.047+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 <== osd.3 v2:172.21.15.72:6808/3190295309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff5c0076f10 con 0x7ff5c0074400 2023-12-12T09:52:40.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.047+0000 7fde47fff640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde4815f260 0x7fde4814b420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde4000efa0 tx=0x7fde4000c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.048+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40014480 con 0x7fde4815f260 2023-12-12T09:52:40.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.048+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4000f040 con 0x7fde4815f260 2023-12-12T09:52:40.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.048+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40014820 con 0x7fde4815f260 2023-12-12T09:52:40.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.048+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4814bed0 con 0x7fde4815f260 2023-12-12T09:52:40.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.048+0000 7fa005cf3640 1 Processor -- start 2023-12-12T09:52:40.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.048+0000 7fa005cf3640 1 -- start start 2023-12-12T09:52:40.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.048+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4815f7c0 con 0x7fde4815f260 2023-12-12T09:52:40.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.049+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde48162d40 con 0x7fde4815f260 2023-12-12T09:52:40.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7f8ac6490640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ac006d1d0 0x7f8ac014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7f8ac6490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac0153900 0x7f8ac01ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7fecf3ba9640 1 Processor -- start 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7f8ac6490640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ac015a580 0x7f8ac00af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7f8ac6490640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01ef8b0 con 0x7f8ac006d1d0 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7f8ac6490640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01ef2f0 con 0x7f8ac015a580 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7f8ac6490640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01ef5d0 con 0x7f8ac0153900 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7f8ab77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac0153900 0x7f8ac01ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f8ab77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac0153900 0x7f8ac01ee8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49424/0 (socket says 172.21.15.72:49424) 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f8ab77fe640 1 -- 172.21.15.72:0/1519657300 learned_addr learned my addr 172.21.15.72:0/1519657300 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f8ab77fe640 1 -- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ac015a580 msgr2=0x7f8ac00af0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f8ab77fe640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ac015a580 0x7f8ac00af0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f8ab77fe640 1 -- 172.21.15.72:0/1519657300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ac006d1d0 msgr2=0x7f8ac014fc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:52:40.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f8ab77fe640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ac006d1d0 0x7f8ac014fc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f8ab77fe640 1 -- 172.21.15.72:0/1519657300 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa800d510 con 0x7f8ac0153900 2023-12-12T09:52:40.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7fecf3ba9640 1 -- start start 2023-12-12T09:52:40.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7fecf3ba9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec153900 0x7fecec070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7fecf3ba9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fecec15a580 0x7fecec06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7fecf3ba9640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fecec06d1d0 0x7fecec0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7fecf3ba9640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec153ce0 con 0x7fecec06d1d0 2023-12-12T09:52:40.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7fecf3ba9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec06d650 con 0x7fecec15a580 2023-12-12T09:52:40.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7fecf3ba9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0cca90 con 0x7fecec153900 2023-12-12T09:52:40.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7f8ab77fe640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac0153900 0x7f8ac01ee8a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8aa802c0f0 tx=0x7f8aa80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7fecf1144640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fecec15a580 0x7fecec06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa803b070 con 0x7f8ac0153900 2023-12-12T09:52:40.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa803c040 con 0x7f8ac0153900 2023-12-12T09:52:40.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8041460 con 0x7f8ac0153900 2023-12-12T09:52:40.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac01efb70 con 0x7f8ac0153900 2023-12-12T09:52:40.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7fecf1144640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fecec15a580 0x7fecec06cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57754/0 (socket says 172.21.15.72:57754) 2023-12-12T09:52:40.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7fecf1144640 1 -- 172.21.15.72:0/997187319 learned_addr learned my addr 172.21.15.72:0/997187319 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7fecf2146640 1 --2- 172.21.15.72:0/997187319 >> v2:172.21.15.72:3300/0 conn(0x7fecec06d1d0 0x7fecec0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7fecf1945640 1 --2- 172.21.15.72:0/997187319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec153900 0x7fecec070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.053+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac0150140 con 0x7f8ac0153900 2023-12-12T09:52:40.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.054+0000 7fecf2146640 1 -- 172.21.15.72:0/997187319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec153900 msgr2=0x7fecec070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.054+0000 7fecf2146640 1 --2- 172.21.15.72:0/997187319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec153900 0x7fecec070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.054+0000 7fecf2146640 1 -- 172.21.15.72:0/997187319 >> v2:172.21.15.113:3300/0 conn(0x7fecec15a580 msgr2=0x7fecec06cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.054+0000 7fecf2146640 1 --2- 172.21.15.72:0/997187319 >> v2:172.21.15.113:3300/0 conn(0x7fecec15a580 0x7fecec06cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.054+0000 7fecf2146640 1 -- 172.21.15.72:0/997187319 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0ccd70 con 0x7fecec06d1d0 2023-12-12T09:52:40.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.054+0000 7fecf1144640 1 --2- 172.21.15.72:0/997187319 >> v2:172.21.15.113:3300/0 conn(0x7fecec15a580 0x7fecec06cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:40.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.054+0000 7fecf1945640 1 --2- 172.21.15.72:0/997187319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec153900 0x7fecec070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:40.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7fa005cf3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 0x7fa00006c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7fa005cf3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa00015e2b0 0x7fa00006cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.050+0000 7fa005cf3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00006ca40 con 0x7fa00006de20 2023-12-12T09:52:40.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.054+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ac00cd330 con 0x7f8ac0153900 2023-12-12T09:52:40.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7fa005cf3640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa000071cd0 con 0x7fa00015e2b0 2023-12-12T09:52:40.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7fa005cf3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa000070f10 con 0x7fa000157a60 2023-12-12T09:52:40.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f9ff6ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa00006de20 0x7fa00015dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54878/0 (socket says 172.21.15.72:54878) 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.051+0000 7f9ff6ffd640 1 -- 172.21.15.72:0/2238688366 learned_addr learned my addr 172.21.15.72:0/2238688366 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.052+0000 7f9ffffff640 1 --2- 172.21.15.72:0/2238688366 >> v2:172.21.15.72:3300/0 conn(0x7fa00015e2b0 0x7fa00006cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.055+0000 7f9ffeffd640 1 -- 172.21.15.72:0/2238688366 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591869933 0 0) 0x7fa00006ca40 con 0x7fa00006de20 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.055+0000 7f9fff7fe640 1 --2- 172.21.15.72:0/2238688366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 0x7fa00006c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.055+0000 7f9ffeffd640 1 -- 172.21.15.72:0/2238688366 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fec003520 con 0x7fa00006de20 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.055+0000 7f9ffffff640 1 -- 172.21.15.72:0/2238688366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 msgr2=0x7fa00006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.055+0000 7f9ffffff640 1 --2- 172.21.15.72:0/2238688366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 0x7fa00006c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.055+0000 7f9ffffff640 1 -- 172.21.15.72:0/2238688366 >> v1:172.21.15.113:6789/0 conn(0x7fa00006de20 legacy=0x7fa00015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.055+0000 7f9ffffff640 1 -- 172.21.15.72:0/2238688366 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000711f0 con 0x7fa00015e2b0 2023-12-12T09:52:40.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.056+0000 7fecf2146640 1 --2- 172.21.15.72:0/997187319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec06d1d0 0x7fecec0b43e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b1a80 tx=0x7fece402c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aaf3e8863b28bf server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.056+0000 7fecf0943640 1 -- 172.21.15.72:0/997187319 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4031610 con 0x7fecec06d1d0 2023-12-12T09:52:40.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.056+0000 7fecf0943640 1 -- 172.21.15.72:0/997187319 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece4031770 con 0x7fecec06d1d0 2023-12-12T09:52:40.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.056+0000 7fecf0943640 1 -- 172.21.15.72:0/997187319 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4031b00 con 0x7fecec06d1d0 2023-12-12T09:52:40.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.056+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde40014980 con 0x7fde4815f260 2023-12-12T09:52:40.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.057+0000 7fecf3ba9640 1 -- 172.21.15.72:0/997187319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec06d1d0 msgr2=0x7fecec0b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.057+0000 7fecf3ba9640 1 --2- 172.21.15.72:0/997187319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec06d1d0 0x7fecec0b43e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b1a80 tx=0x7fece402c950 comp rx=0 tx=0).stop 2023-12-12T09:52:40.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.058+0000 7fde2ffff640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fde3406af40 0x7fde3406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.058+0000 7fecf3ba9640 1 -- 172.21.15.72:0/997187319 shutdown_connections 2023-12-12T09:52:40.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.058+0000 7fecf3ba9640 1 --2- 172.21.15.72:0/997187319 >> v2:172.21.15.113:3300/0 conn(0x7fecec15a580 0x7fecec06cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.058+0000 7fecf3ba9640 1 --2- 172.21.15.72:0/997187319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec06d1d0 0x7fecec0b43e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.058+0000 7fecf3ba9640 1 --2- 172.21.15.72:0/997187319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec153900 0x7fecec070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.058+0000 7fecf3ba9640 1 -- 172.21.15.72:0/997187319 >> 172.21.15.72:0/997187319 conn(0x7fecec0b0a90 msgr2=0x7fecec0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.057+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8aa8037400 con 0x7f8ac0153900 2023-12-12T09:52:40.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.057+0000 7f8abd7fa640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8aa006ae90 0x7f8aa006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.058+0000 7fecf3ba9640 1 -- 172.21.15.72:0/997187319 shutdown_connections 2023-12-12T09:52:40.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7f8abffff640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8aa006ae90 0x7f8aa006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.058+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fde4008b090 con 0x7fde4815f260 2023-12-12T09:52:40.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7fde477fe640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fde3406af40 0x7fde3406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7fecf3ba9640 1 -- 172.21.15.72:0/997187319 wait complete. 2023-12-12T09:52:40.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8aa807ae60 con 0x7f8ac0153900 2023-12-12T09:52:40.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7f8abd7fa640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7f8aa0075170 0x7f8aa0077610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 --> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8aa0077ce0 con 0x7f8aa0075170 2023-12-12T09:52:40.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aa807ac30 con 0x7f8ac0153900 2023-12-12T09:52:40.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7f9ffffff640 1 --2- 172.21.15.72:0/2238688366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00015e2b0 0x7fa00006cca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80097a0 tx=0x7f9ff802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b8eeed5dc2e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7fde2ffff640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fde34074580 0x7fde340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.060+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 --> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde34077090 con 0x7fde34074580 2023-12-12T09:52:40.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.060+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde4008b440 con 0x7fde4815f260 2023-12-12T09:52:40.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.059+0000 7f9ffeffd640 1 -- 172.21.15.72:0/2238688366 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8031710 con 0x7fa00015e2b0 2023-12-12T09:52:40.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.060+0000 7f8ac4a2d640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7f8aa0075170 0x7f8aa0077610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.060+0000 7f9ffeffd640 1 -- 172.21.15.72:0/2238688366 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff802f050 con 0x7fa00015e2b0 2023-12-12T09:52:40.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.060+0000 7f9ffeffd640 1 -- 172.21.15.72:0/2238688366 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8031710 con 0x7fa00015e2b0 2023-12-12T09:52:40.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fde46ffd640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fde34074580 0x7fde340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7f8ac4a2d640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7f8aa0075170 0x7f8aa0077610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fde477fe640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fde3406af40 0x7fde3406d400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde38005880 tx=0x7fde38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fa005cf3640 1 -- 172.21.15.72:0/2238688366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00015e2b0 msgr2=0x7fa00006cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fa005cf3640 1 --2- 172.21.15.72:0/2238688366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00015e2b0 0x7fa00006cca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80097a0 tx=0x7f9ff802b7b0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fa005cf3640 1 -- 172.21.15.72:0/2238688366 shutdown_connections 2023-12-12T09:52:40.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fa005cf3640 1 --2- 172.21.15.72:0/2238688366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00015e2b0 0x7fa00006cca0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fa005cf3640 1 --2- 172.21.15.72:0/2238688366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 0x7fa00006c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fa005cf3640 1 -- 172.21.15.72:0/2238688366 >> 172.21.15.72:0/2238688366 conn(0x7fa0000b0a90 msgr2=0x7fa0000b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7f8abffff640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8aa006ae90 0x7f8aa006d350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ac01532a0 tx=0x7f8ab0007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fa005cf3640 1 -- 172.21.15.72:0/2238688366 shutdown_connections 2023-12-12T09:52:40.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.061+0000 7fa005cf3640 1 -- 172.21.15.72:0/2238688366 wait complete. 2023-12-12T09:52:40.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.062+0000 7fde46ffd640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fde34074580 0x7fde340769c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.063+0000 7fecf3ba9640 1 Processor -- start 2023-12-12T09:52:40.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.063+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 <== osd.4 v2:172.21.15.113:6800/1025641669 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8aa0077ce0 con 0x7f8aa0075170 2023-12-12T09:52:40.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.064+0000 7fecf3ba9640 1 -- start start 2023-12-12T09:52:40.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.064+0000 7fa005cf3640 1 Processor -- start 2023-12-12T09:52:40.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.064+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 <== osd.5 v2:172.21.15.113:6808/1716073726 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fde34077090 con 0x7fde34074580 2023-12-12T09:52:40.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.064+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 --> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ee4050170 con 0x7f9ed00743e0 2023-12-12T09:52:40.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.064+0000 7fa005cf3640 1 -- start start 2023-12-12T09:52:40.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.066+0000 7fecf3ba9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec06d1d0 0x7fecec1423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.066+0000 7fecf3ba9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecec153900 0x7fecec0b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.066+0000 7fecf3ba9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec15a580 0x7fecec0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.066+0000 7fecf2146640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec15a580 0x7fecec0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.067+0000 7fecf2146640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec15a580 0x7fecec0b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40382/0 (socket says 172.21.15.72:40382) 2023-12-12T09:52:40.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.067+0000 7fecf1144640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecec153900 0x7fecec0b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.067+0000 7fecf1144640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecec153900 0x7fecec0b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57756/0 (socket says 172.21.15.72:57756) 2023-12-12T09:52:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.067+0000 7fecf1945640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec06d1d0 0x7fecec1423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.067+0000 7fecf2146640 1 -- 172.21.15.72:0/1996030441 learned_addr learned my addr 172.21.15.72:0/1996030441 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.067+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec154b70 con 0x7fecec15a580 2023-12-12T09:52:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.067+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec154e50 con 0x7fecec153900 2023-12-12T09:52:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.067+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec155130 con 0x7fecec06d1d0 2023-12-12T09:52:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.068+0000 7f9ee27fc640 1 -- 172.21.15.72:0/1295220659 <== osd.1 v2:172.21.15.72:6800/1034360867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9eec04f090 con 0x7f9ed00743e0 2023-12-12T09:52:40.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.068+0000 7fecf1144640 1 -- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec06d1d0 msgr2=0x7fecec1423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.068+0000 7fecf1144640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecec06d1d0 0x7fecec1423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.068+0000 7fecf1144640 1 -- 172.21.15.72:0/1996030441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec15a580 msgr2=0x7fecec0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.068+0000 7fecf1144640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecec15a580 0x7fecec0b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.068+0000 7fecf1144640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece4009400 con 0x7fecec153900 2023-12-12T09:52:40.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7fcc3a7fc640 1 -- 172.21.15.72:0/825903993 --> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc541593a0 con 0x7fcc40074550 2023-12-12T09:52:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7fecf1144640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecec153900 0x7fecec0b2bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00e810 tx=0x7fecdc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f9ed00743e0 msgr2=0x7f9ed0076820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7f9ef2b35640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f9ed00743e0 0x7f9ed0076820 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ed006aea0 msgr2=0x7f9ed006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7f9ef2b35640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ed006aea0 0x7f9ed006d360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9edc001010 tx=0x7f9edc009040 comp rx=0 tx=0).stop 2023-12-12T09:52:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee40067e0 msgr2=0x7f9ee4036810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7f9ef2b35640 1 --2- 172.21.15.72:0/1295220659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee40067e0 0x7f9ee4036810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8033000 tx=0x7f9ee8004730 comp rx=0 tx=0).stop 2023-12-12T09:52:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7f9ef2334640 1 -- 172.21.15.72:0/1295220659 reap_dead start 2023-12-12T09:52:40.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7fcc50ff9640 1 -- 172.21.15.72:0/825903993 <== osd.0 v2:172.21.15.72:6823/2303781576 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc4c007270 con 0x7fcc40074550 2023-12-12T09:52:40.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 shutdown_connections 2023-12-12T09:52:40.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 >> 172.21.15.72:0/1295220659 conn(0x7f9ee4009f80 msgr2=0x7f9ee404db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7fa005cf3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00006de20 0x7fa0001488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7fa005cf3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 0x7fa000151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7fa005cf3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00015e2b0 0x7fa0000b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7fa005cf3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000148e00 con 0x7fa00006de20 2023-12-12T09:52:40.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7fa005cf3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000156a50 con 0x7fa00015e2b0 2023-12-12T09:52:40.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.069+0000 7fa005cf3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa000155e40 con 0x7fa000157a60 2023-12-12T09:52:40.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7f9ffffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00015e2b0 0x7fa0000b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 shutdown_connections 2023-12-12T09:52:40.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7f9ef2b35640 1 -- 172.21.15.72:0/1295220659 wait complete. 2023-12-12T09:52:40.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7f9ff6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 0x7fa000151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00c6d0 con 0x7fecec153900 2023-12-12T09:52:40.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc00c830 con 0x7fecec153900 2023-12-12T09:52:40.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00cba0 con 0x7fecec153900 2023-12-12T09:52:40.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec155410 con 0x7fecec153900 2023-12-12T09:52:40.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec155c10 con 0x7fecec153900 2023-12-12T09:52:40.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7f9ffffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00015e2b0 0x7fa0000b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57758/0 (socket says 172.21.15.72:57758) 2023-12-12T09:52:40.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.070+0000 7f9ffffff640 1 -- 172.21.15.72:0/3979095672 learned_addr learned my addr 172.21.15.72:0/3979095672 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7fcc40074550 msgr2=0x7fcc40076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7fcc59175640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7fcc40074550 0x7fcc40076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7f9fff7fe640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00006de20 0x7fa0001488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc4006af10 msgr2=0x7fcc4006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7fcc59175640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc4006af10 0x7fcc4006d3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc48036000 tx=0x7fcc4803d040 comp rx=0 tx=0).stop 2023-12-12T09:52:40.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc540b2700 msgr2=0x7fcc540cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7fcc59175640 1 --2- 172.21.15.72:0/825903993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc540b2700 0x7fcc540cae70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400bf90 tx=0x7fcc44009f10 comp rx=0 tx=0).stop 2023-12-12T09:52:40.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7f9ffffff640 1 -- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 msgr2=0x7fa000151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7f9ffffff640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa000157a60 0x7fa000151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.072+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fecec159650 con 0x7fecec153900 2023-12-12T09:52:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.071+0000 7f9ffffff640 1 -- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00006de20 msgr2=0x7fa0001488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.072+0000 7fcc58974640 1 -- 172.21.15.72:0/825903993 reap_dead start 2023-12-12T09:52:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.072+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 shutdown_connections 2023-12-12T09:52:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.072+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 >> 172.21.15.72:0/825903993 conn(0x7fcc5406a460 msgr2=0x7fcc541569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.072+0000 7f9ffffff640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00006de20 0x7fa0001488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.072+0000 7f9ffffff640 1 -- 172.21.15.72:0/3979095672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff8009400 con 0x7fa00015e2b0 2023-12-12T09:52:40.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.072+0000 7f9fff7fe640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa00006de20 0x7fa0001488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:40.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.074+0000 7f9ffffff640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00015e2b0 0x7fa0000b3240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80098c0 tx=0x7f9ff80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.074+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 shutdown_connections 2023-12-12T09:52:40.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.074+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 --> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f819c005bc0 con 0x7f819c0015e0 2023-12-12T09:52:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.074+0000 7fcc59175640 1 -- 172.21.15.72:0/825903993 wait complete. 2023-12-12T09:52:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.075+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8031710 con 0x7fa00015e2b0 2023-12-12T09:52:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.075+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff8031890 con 0x7fa00015e2b0 2023-12-12T09:52:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.075+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8031c40 con 0x7fa00015e2b0 2023-12-12T09:52:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.075+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecdc00cd00 con 0x7fecec153900 2023-12-12T09:52:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.075+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa000156120 con 0x7fa00015e2b0 2023-12-12T09:52:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.075+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0001566e0 con 0x7fa00015e2b0 2023-12-12T09:52:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.076+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa00006e280 con 0x7fa00015e2b0 2023-12-12T09:52:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.078+0000 7f81a37fe640 1 -- 172.21.15.72:0/3730282001 <== osd.2 v2:172.21.15.72:6816/323046314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f819c005bc0 con 0x7f819c0015e0 2023-12-12T09:52:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.079+0000 7fece27fc640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fecd006af10 0x7fecd006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.080+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ff803a040 con 0x7fa00015e2b0 2023-12-12T09:52:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.080+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fecdc089b20 con 0x7fecec153900 2023-12-12T09:52:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.080+0000 7fece27fc640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7fecd0074550 0x7fecd0076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.080+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecd0077060 con 0x7fecd0074550 2023-12-12T09:52:40.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.080+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecdc089ed0 con 0x7fecec153900 2023-12-12T09:52:40.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.080+0000 7fecf1945640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fecd006af10 0x7fecd006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.081+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7f819c0015e0 msgr2=0x7f819c003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.081+0000 7f81c1789640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7f819c0015e0 0x7f819c003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.081+0000 7fecf2146640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7fecd0074550 0x7fecd0076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.082+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f81a806ae40 msgr2=0x7f81a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.082+0000 7f81c1789640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f81a806ae40 0x7f81a806d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81b00011e0 tx=0x7f81b0007b70 comp rx=0 tx=0).stop 2023-12-12T09:52:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.082+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc0708d0 msgr2=0x7f81bc1ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.082+0000 7f81c1789640 1 --2- 172.21.15.72:0/3730282001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81bc0708d0 0x7f81bc1ff9b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c4a0 tx=0x7f81b400c960 comp rx=0 tx=0).stop 2023-12-12T09:52:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.082+0000 7f81bb7fe640 1 -- 172.21.15.72:0/3730282001 reap_dead start 2023-12-12T09:52:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.082+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 shutdown_connections 2023-12-12T09:52:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.082+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 >> 172.21.15.72:0/3730282001 conn(0x7f81bc0b0a90 msgr2=0x7f81bc162dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.081+0000 7f9ffcff9640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9fe006aec0 0x7f9fe006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.082+0000 7f9fff7fe640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9fe006aec0 0x7f9fe006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.083+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 --> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5c4002cb0 con 0x7ff5c0074400 2023-12-12T09:52:40.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.083+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 shutdown_connections 2023-12-12T09:52:40.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.083+0000 7f81c1789640 1 -- 172.21.15.72:0/3730282001 wait complete. 2023-12-12T09:52:40.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.083+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ff80044c0 con 0x7fa00015e2b0 2023-12-12T09:52:40.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.083+0000 7f9ffcff9640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f9fe00751a0 0x7f9fe0077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.083+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 --> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fe0077d10 con 0x7f9fe00751a0 2023-12-12T09:52:40.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.083+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ff807a2b0 con 0x7fa00015e2b0 2023-12-12T09:52:40.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.084+0000 7fecf1945640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fecd006af10 0x7fecd006d3d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fece800d620 tx=0x7fece8007810 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.084+0000 7f9fff7fe640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9fe006aec0 0x7f9fe006d380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0007330 tx=0x7f9ff000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.084+0000 7f9ff6ffd640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f9fe00751a0 0x7f9fe0077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.084+0000 7ff5d27fc640 1 -- 172.21.15.72:0/2792713820 <== osd.3 v2:172.21.15.72:6808/3190295309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff5c4002cb0 con 0x7ff5c0074400 2023-12-12T09:52:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.084+0000 7fecf2146640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7fecd0074550 0x7fecd0076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7ff5c0074400 msgr2=0x7ff5c0076840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e3e7c640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7ff5c0074400 0x7ff5c0076840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5c006ae40 msgr2=0x7ff5c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e3e7c640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5c006ae40 0x7ff5c006d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc036000 tx=0x7ff5cc030000 comp rx=0 tx=0).stop 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5dc06cc90 msgr2=0x7ff5dc16cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e3e7c640 1 --2- 172.21.15.72:0/2792713820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5dc06cc90 0x7ff5dc16cf00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c400 tx=0x7ff5d800c8c0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e2419640 1 -- 172.21.15.72:0/2792713820 reap_dead start 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 shutdown_connections 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.085+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 >> 172.21.15.72:0/2792713820 conn(0x7ff5dc0b0a90 msgr2=0x7ff5dc0aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.086+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 shutdown_connections 2023-12-12T09:52:40.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.086+0000 7f9ff6ffd640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f9fe00751a0 0x7f9fe0077640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.086+0000 7ff5e3e7c640 1 -- 172.21.15.72:0/2792713820 wait complete. 2023-12-12T09:52:40.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.087+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 <== osd.6 v2:172.21.15.113:6816/3330758472 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fecd0077060 con 0x7fecd0074550 2023-12-12T09:52:40.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.088+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 --> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ac00cd050 con 0x7f8aa0075170 2023-12-12T09:52:40.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.088+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 <== osd.7 v2:172.21.15.113:6824/1904806807 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9fe0077d10 con 0x7f9fe00751a0 2023-12-12T09:52:40.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.089+0000 7f8abd7fa640 1 -- 172.21.15.72:0/1519657300 <== osd.4 v2:172.21.15.113:6800/1025641669 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8ab8007cd0 con 0x7f8aa0075170 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.093+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7f8aa0075170 msgr2=0x7f8aa0077610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.093+0000 7f8ac6490640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7f8aa0075170 0x7f8aa0077610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.093+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8aa006ae90 msgr2=0x7f8aa006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.093+0000 7f8ac6490640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8aa006ae90 0x7f8aa006d350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ac01532a0 tx=0x7f8ab0007d50 comp rx=0 tx=0).stop 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.093+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac0153900 msgr2=0x7f8ac01ee8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.093+0000 7f8ac6490640 1 --2- 172.21.15.72:0/1519657300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ac0153900 0x7f8ac01ee8a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8aa802c0f0 tx=0x7f8aa80046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.093+0000 7f8ac4a2d640 1 -- 172.21.15.72:0/1519657300 reap_dead start 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.094+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 shutdown_connections 2023-12-12T09:52:40.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.094+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 >> 172.21.15.72:0/1519657300 conn(0x7f8ac00b0a90 msgr2=0x7f8ac00aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.100 INFO:teuthology.orchestra.run.smithi072.stdout:55834574857 2023-12-12T09:52:40.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.095+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 shutdown_connections 2023-12-12T09:52:40.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.095+0000 7f8ac6490640 1 -- 172.21.15.72:0/1519657300 wait complete. 2023-12-12T09:52:40.101 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-12T09:52:40.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.095+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 --> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fde480cd7f0 con 0x7fde34074580 2023-12-12T09:52:40.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.096+0000 7fde2ffff640 1 -- 172.21.15.72:0/1672888712 <== osd.5 v2:172.21.15.113:6808/1716073726 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fde3c041040 con 0x7fde34074580 2023-12-12T09:52:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.097+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fde34074580 msgr2=0x7fde340769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.097+0000 7fde4dd78640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fde34074580 0x7fde340769c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.098+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fde3406af40 msgr2=0x7fde3406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.098+0000 7fde4dd78640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fde3406af40 0x7fde3406d400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde38005880 tx=0x7fde38009040 comp rx=0 tx=0).stop 2023-12-12T09:52:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.098+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde4815f260 msgr2=0x7fde4814b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.098+0000 7fde4dd78640 1 --2- 172.21.15.72:0/1672888712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde4815f260 0x7fde4814b420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde4000efa0 tx=0x7fde4000c550 comp rx=0 tx=0).stop 2023-12-12T09:52:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.098+0000 7fde47fff640 1 -- 172.21.15.72:0/1672888712 reap_dead start 2023-12-12T09:52:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.098+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 shutdown_connections 2023-12-12T09:52:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.099+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 >> 172.21.15.72:0/1672888712 conn(0x7fde480b0a90 msgr2=0x7fde4814e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.099+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 shutdown_connections 2023-12-12T09:52:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.099+0000 7fde4dd78640 1 -- 172.21.15.72:0/1672888712 wait complete. 2023-12-12T09:52:40.104 INFO:teuthology.orchestra.run.smithi072.stdout:55834574857 2023-12-12T09:52:40.104 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-12T09:52:40.108 INFO:teuthology.orchestra.run.smithi072.stdout:55834574857 2023-12-12T09:52:40.108 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-12T09:52:40.111 INFO:teuthology.orchestra.run.smithi072.stdout:55834574857 2023-12-12T09:52:40.111 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-12T09:52:40.119 INFO:teuthology.orchestra.run.smithi072.stdout:55834574859 2023-12-12T09:52:40.119 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-12T09:52:40.120 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-12-12T09:52:40.121 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-12T09:52:40.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.122+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 --> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fecec0cd330 con 0x7fecd0074550 2023-12-12T09:52:40.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.123+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 --> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0000717b0 con 0x7f9fe00751a0 2023-12-12T09:52:40.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.123+0000 7fece27fc640 1 -- 172.21.15.72:0/1996030441 <== osd.6 v2:172.21.15.113:6816/3330758472 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fece4004040 con 0x7fecd0074550 2023-12-12T09:52:40.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.123+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7fecd0074550 msgr2=0x7fecd0076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.123+0000 7fecf3ba9640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7fecd0074550 0x7fecd0076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fecd006af10 msgr2=0x7fecd006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf3ba9640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fecd006af10 0x7fecd006d3d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fece800d620 tx=0x7fece8007810 comp rx=0 tx=0).stop 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecec153900 msgr2=0x7fecec0b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf3ba9640 1 --2- 172.21.15.72:0/1996030441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecec153900 0x7fecec0b2bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00e810 tx=0x7fecdc00ecd0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf2146640 1 -- 172.21.15.72:0/1996030441 reap_dead start 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 shutdown_connections 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 >> 172.21.15.72:0/1996030441 conn(0x7fecec0b0a90 msgr2=0x7fecec0af310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 shutdown_connections 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7fecf3ba9640 1 -- 172.21.15.72:0/1996030441 wait complete. 2023-12-12T09:52:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.124+0000 7f9ffcff9640 1 -- 172.21.15.72:0/3979095672 <== osd.7 v2:172.21.15.113:6824/1904806807 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9fe8014040 con 0x7f9fe00751a0 2023-12-12T09:52:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f9fe00751a0 msgr2=0x7f9fe0077640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f9fe00751a0 0x7f9fe0077640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9fe006aec0 msgr2=0x7f9fe006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9fe006aec0 0x7f9fe006d380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0007330 tx=0x7f9ff000b040 comp rx=0 tx=0).stop 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00015e2b0 msgr2=0x7fa0000b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 --2- 172.21.15.72:0/3979095672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00015e2b0 0x7fa0000b3240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80098c0 tx=0x7f9ff80046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7f9ffffff640 1 -- 172.21.15.72:0/3979095672 reap_dead start 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 shutdown_connections 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 >> 172.21.15.72:0/3979095672 conn(0x7fa0000b0a90 msgr2=0x7fa0000700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 shutdown_connections 2023-12-12T09:52:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.125+0000 7fa005cf3640 1 -- 172.21.15.72:0/3979095672 wait complete. 2023-12-12T09:52:40.147 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-12-12T09:52:40.147 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-12T09:52:40.149 INFO:teuthology.orchestra.run.smithi072.stdout:55834574859 2023-12-12T09:52:40.149 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-12T09:52:40.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.368+0000 7f1bba197640 1 Processor -- start 2023-12-12T09:52:40.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.369+0000 7f1bba197640 1 -- start start 2023-12-12T09:52:40.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.372+0000 7f1bba197640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1bac03e810 0x7f1bac053bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.372+0000 7f1bb37fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1bac0e3090 0x7f1bac057d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54894/0 (socket says 172.21.15.72:54894) 2023-12-12T09:52:40.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.372+0000 7f1bb37fe640 1 -- 172.21.15.72:0/1270328840 learned_addr learned my addr 172.21.15.72:0/1270328840 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.373+0000 7f1bba197640 1 --2- 172.21.15.72:0/1270328840 >> v2:172.21.15.113:3300/0 conn(0x7f1bac054100 0x7f1bac0068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.373+0000 7f1bba197640 1 -- 172.21.15.72:0/1270328840 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bac058540 con 0x7f1bac0e3090 2023-12-12T09:52:40.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.373+0000 7f1bba197640 1 -- 172.21.15.72:0/1270328840 --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac006170 con 0x7f1bac03e810 2023-12-12T09:52:40.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.374+0000 7f1bba197640 1 -- 172.21.15.72:0/1270328840 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac054570 con 0x7f1bac054100 2023-12-12T09:52:40.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.374+0000 7f1bb2ffd640 1 --2- 172.21.15.72:0/1270328840 >> v2:172.21.15.72:3300/0 conn(0x7f1bac03e810 0x7f1bac053bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.374+0000 7f1bb27fc640 1 -- 172.21.15.72:0/1270328840 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897449230 0 0) 0x7f1bac058540 con 0x7f1bac0e3090 2023-12-12T09:52:40.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.374+0000 7f1bb27fc640 1 -- 172.21.15.72:0/1270328840 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba4003520 con 0x7f1bac0e3090 2023-12-12T09:52:40.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.375+0000 7f1bb3fff640 1 --2- 172.21.15.72:0/1270328840 >> v2:172.21.15.113:3300/0 conn(0x7f1bac054100 0x7f1bac0068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.375+0000 7f1bb27fc640 1 -- 172.21.15.72:0/1270328840 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 454846641 0 0) 0x7f1ba4003520 con 0x7f1bac0e3090 2023-12-12T09:52:40.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.379+0000 7f1bb2ffd640 1 -- 172.21.15.72:0/1270328840 >> v2:172.21.15.113:3300/0 conn(0x7f1bac054100 msgr2=0x7f1bac0068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.379+0000 7f1bb2ffd640 1 --2- 172.21.15.72:0/1270328840 >> v2:172.21.15.113:3300/0 conn(0x7f1bac054100 0x7f1bac0068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.379+0000 7f1bb2ffd640 1 -- 172.21.15.72:0/1270328840 >> v1:172.21.15.113:6789/0 conn(0x7f1bac0e3090 legacy=0x7f1bac057d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.379+0000 7f1bb2ffd640 1 -- 172.21.15.72:0/1270328840 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac058820 con 0x7f1bac03e810 2023-12-12T09:52:40.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.381+0000 7f1bb2ffd640 1 --2- 172.21.15.72:0/1270328840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac053bc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0e7ca0 tx=0x7f1ba802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cf939604881f1d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.381+0000 7f1bb27fc640 1 -- 172.21.15.72:0/1270328840 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0001670 con 0x7f1bac03e810 2023-12-12T09:52:40.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.382+0000 7f1bb27fc640 1 -- 172.21.15.72:0/1270328840 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8031520 con 0x7f1bac03e810 2023-12-12T09:52:40.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.383+0000 7f1bba197640 1 -- 172.21.15.72:0/1270328840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 msgr2=0x7f1bac053bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.383+0000 7f1bba197640 1 --2- 172.21.15.72:0/1270328840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac053bc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0e7ca0 tx=0x7f1ba802b840 comp rx=0 tx=0).stop 2023-12-12T09:52:40.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.383+0000 7f1bba197640 1 -- 172.21.15.72:0/1270328840 shutdown_connections 2023-12-12T09:52:40.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.384+0000 7f1bba197640 1 --2- 172.21.15.72:0/1270328840 >> v2:172.21.15.113:3300/0 conn(0x7f1bac054100 0x7f1bac0068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.384+0000 7f1bba197640 1 --2- 172.21.15.72:0/1270328840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac053bc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.384+0000 7f1bba197640 1 -- 172.21.15.72:0/1270328840 >> 172.21.15.72:0/1270328840 conn(0x7f1bac0559b0 msgr2=0x7f1bac055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.384+0000 7f1bba197640 1 -- 172.21.15.72:0/1270328840 shutdown_connections 2023-12-12T09:52:40.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.384+0000 7f1bba197640 1 -- 172.21.15.72:0/1270328840 wait complete. 2023-12-12T09:52:40.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.387+0000 7f1bba197640 1 Processor -- start 2023-12-12T09:52:40.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.388+0000 7f1bba197640 1 -- start start 2023-12-12T09:52:40.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.389+0000 7f1bba197640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac04f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.389+0000 7f1bba197640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bac054100 0x7f1bac03cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.389+0000 7f1bba197640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bac0e3090 0x7f1bac03d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.389+0000 7f1bba197640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac052e90 con 0x7f1bac03e810 2023-12-12T09:52:40.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.389+0000 7f1bba197640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac052280 con 0x7f1bac054100 2023-12-12T09:52:40.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.389+0000 7f1bba197640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac052560 con 0x7f1bac0e3090 2023-12-12T09:52:40.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.390+0000 7f1bb2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bac054100 0x7f1bac03cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.391+0000 7f1bb3fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bac0e3090 0x7f1bac03d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.391+0000 7f1bb37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac04f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.391+0000 7f1bb37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac04f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40412/0 (socket says 172.21.15.72:40412) 2023-12-12T09:52:40.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.391+0000 7f1bb37fe640 1 -- 172.21.15.72:0/3573018097 learned_addr learned my addr 172.21.15.72:0/3573018097 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.391+0000 7fae51153640 1 Processor -- start 2023-12-12T09:52:40.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.392+0000 7f1bb37fe640 1 -- 172.21.15.72:0/3573018097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bac0e3090 msgr2=0x7f1bac03d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.392+0000 7f1bb37fe640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bac0e3090 0x7f1bac03d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.392+0000 7f1bb37fe640 1 -- 172.21.15.72:0/3573018097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bac054100 msgr2=0x7f1bac03cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.392+0000 7f1bb37fe640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bac054100 0x7f1bac03cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.392+0000 7f1bb37fe640 1 -- 172.21.15.72:0/3573018097 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8009400 con 0x7f1bac03e810 2023-12-12T09:52:40.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.392+0000 7f1bb37fe640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac04f0b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000caf0 tx=0x7f1ba000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.393+0000 7f1b9ffff640 1 -- 172.21.15.72:0/3573018097 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0005ce0 con 0x7f1bac03e810 2023-12-12T09:52:40.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.393+0000 7f1b9ffff640 1 -- 172.21.15.72:0/3573018097 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba0005e40 con 0x7f1bac03e810 2023-12-12T09:52:40.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.393+0000 7f1b9ffff640 1 -- 172.21.15.72:0/3573018097 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0005ce0 con 0x7f1bac03e810 2023-12-12T09:52:40.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.392+0000 7fae51153640 1 -- start start 2023-12-12T09:52:40.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.394+0000 7fae51153640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440df420 0x7fae44053260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.394+0000 7fae51153640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae440392d0 0x7fae440537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.394+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac052840 con 0x7f1bac03e810 2023-12-12T09:52:40.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.395+0000 7fae4ad9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fae4404fa90 0x7fae4404fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54904/0 (socket says 172.21.15.72:54904) 2023-12-12T09:52:40.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.395+0000 7fae4b59e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae440392d0 0x7fae440537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.395+0000 7fae4b59e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae440392d0 0x7fae440537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57780/0 (socket says 172.21.15.72:57780) 2023-12-12T09:52:40.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.395+0000 7fae4ad9d640 1 -- 172.21.15.72:0/3474183704 learned_addr learned my addr 172.21.15.72:0/3474183704 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.395+0000 7fae51153640 1 -- 172.21.15.72:0/3474183704 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440068a0 con 0x7fae4404fa90 2023-12-12T09:52:40.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.396+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac04f5f0 con 0x7f1bac03e810 2023-12-12T09:52:40.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.396+0000 7fae51153640 1 -- 172.21.15.72:0/3474183704 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae44007770 con 0x7fae440392d0 2023-12-12T09:52:40.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.396+0000 7fae51153640 1 -- 172.21.15.72:0/3474183704 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae44006af0 con 0x7fae440df420 2023-12-12T09:52:40.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.396+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628193200 0 0) 0x7fae440068a0 con 0x7fae4404fa90 2023-12-12T09:52:40.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.396+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae34003520 con 0x7fae4404fa90 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.398+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 796739776 0 0) 0x7fae34003520 con 0x7fae4404fa90 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.398+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440df420 msgr2=0x7fae44053260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.398+0000 7fae49d9b640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440df420 0x7fae44053260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.398+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 >> v2:172.21.15.113:3300/0 conn(0x7fae440392d0 msgr2=0x7fae440537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.398+0000 7fae49d9b640 1 --2- 172.21.15.72:0/3474183704 >> v2:172.21.15.113:3300/0 conn(0x7fae440392d0 0x7fae440537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.398+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae44006dd0 con 0x7fae4404fa90 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.398+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fae4c04ed60 con 0x7fae4404fa90 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.398+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 >> v1:172.21.15.113:6789/0 conn(0x7fae4404fa90 legacy=0x7fae4404fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.399+0000 7fae49d9b640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae34003970 0x7fae34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.399+0000 7fae49d9b640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34004310 0x7fae340047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.399+0000 7fae49d9b640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440df420 0x7fae34005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.399+0000 7f1b9ffff640 1 -- 172.21.15.72:0/3573018097 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ba0007070 con 0x7f1bac03e810 2023-12-12T09:52:40.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.400+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44007770 con 0x7fae34004310 2023-12-12T09:52:40.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.400+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac0057b0 con 0x7f1bac03e810 2023-12-12T09:52:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.401+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44006af0 con 0x7fae34003970 2023-12-12T09:52:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.401+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3400ba40 con 0x7fae440df420 2023-12-12T09:52:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.401+0000 7fae4b59e640 1 --2- 172.21.15.72:0/3474183704 >> v2:172.21.15.113:3300/0 conn(0x7fae440392d0 0x7fae440537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.401+0000 7fae4ad9d640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae34003970 0x7fae34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.402+0000 7fae4a59c640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34004310 0x7fae340047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.402+0000 7fae4b59e640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440df420 0x7fae34005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.401+0000 7f1b9ffff640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.402+0000 7fae4ad9d640 1 -- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440df420 msgr2=0x7fae34005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.403+0000 7f1bb2ffd640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.402+0000 7fae4ad9d640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440df420 0x7fae34005040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.402+0000 7fae4ad9d640 1 -- 172.21.15.72:0/3474183704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34004310 msgr2=0x7fae340047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.405+0000 7fe75862b640 1 Processor -- start 2023-12-12T09:52:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.405+0000 7fe75862b640 1 -- start start 2023-12-12T09:52:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.405+0000 7f03e7ed0640 1 Processor -- start 2023-12-12T09:52:40.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.403+0000 7fae4ad9d640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34004310 0x7fae340047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.403+0000 7fae4ad9d640 1 -- 172.21.15.72:0/3474183704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34003520 con 0x7fae34003970 2023-12-12T09:52:40.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.405+0000 7fae4ad9d640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae34003970 0x7fae34003d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae4c069c20 tx=0x7fae340057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce46eee3ecc7bbb server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.405+0000 7fae4a59c640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34004310 0x7fae340047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:40.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.405+0000 7f1b9ffff640 1 -- 172.21.15.72:0/3573018097 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ba00575a0 con 0x7f1bac03e810 2023-12-12T09:52:40.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.405+0000 7f1bb2ffd640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1b9806aed0 0x7f1b9806d390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8035000 tx=0x7f1ba8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.406+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c06a460 con 0x7fae34003970 2023-12-12T09:52:40.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.406+0000 7f03e7ed0640 1 -- start start 2023-12-12T09:52:40.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.406+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c0ab070 con 0x7fae34003970 2023-12-12T09:52:40.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.407+0000 7fae49d9b640 1 -- 172.21.15.72:0/3474183704 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c06a920 con 0x7fae34003970 2023-12-12T09:52:40.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.407+0000 7fae51153640 1 -- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae34003970 msgr2=0x7fae34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.407+0000 7fae51153640 1 --2- 172.21.15.72:0/3474183704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae34003970 0x7fae34003d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae4c069c20 tx=0x7fae340057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.407+0000 7fae4b59e640 1 -- 172.21.15.72:0/3474183704 reap_dead start 2023-12-12T09:52:40.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.407+0000 7fae51153640 1 -- 172.21.15.72:0/3474183704 shutdown_connections 2023-12-12T09:52:40.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.407+0000 7fae51153640 1 -- 172.21.15.72:0/3474183704 >> 172.21.15.72:0/3474183704 conn(0x7fae44009fb0 msgr2=0x7fae4400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.407+0000 7fae51153640 1 -- 172.21.15.72:0/3474183704 shutdown_connections 2023-12-12T09:52:40.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.408+0000 7fae51153640 1 -- 172.21.15.72:0/3474183704 wait complete. 2023-12-12T09:52:40.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.408+0000 7fe75862b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75015abc0 0x7fe750070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.408+0000 7fe75862b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe7500712a0 0x7fe750071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.408+0000 7fe75862b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe750160ad0 con 0x7fe750143730 2023-12-12T09:52:40.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.408+0000 7fe75862b640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75015b4d0 con 0x7fe7500712a0 2023-12-12T09:52:40.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.408+0000 7fe75862b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe750070280 con 0x7fe75015abc0 2023-12-12T09:52:40.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.408+0000 7f03e7ed0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e00cb600 0x7f03e00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.409+0000 7f03e7ed0640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f03e006d230 0x7f03e006d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.409+0000 7f03e7ed0640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e00ce9f0 con 0x7f03e006c010 2023-12-12T09:52:40.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.409+0000 7f03e7ed0640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e00cdac0 con 0x7f03e006d230 2023-12-12T09:52:40.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.409+0000 7fe755bc6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75015abc0 0x7fe750070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.409+0000 7fe755bc6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75015abc0 0x7fe750070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49472/0 (socket says 172.21.15.72:49472) 2023-12-12T09:52:40.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.409+0000 7fe755bc6640 1 -- 172.21.15.72:0/73303839 learned_addr learned my addr 172.21.15.72:0/73303839 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7f03e5c6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e00cb600 0x7f03e00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7f03e5c6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e00cb600 0x7f03e00cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49466/0 (socket says 172.21.15.72:49466) 2023-12-12T09:52:40.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7f03e5c6c640 1 -- 172.21.15.72:0/4177164859 learned_addr learned my addr 172.21.15.72:0/4177164859 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7f03e7ed0640 1 -- 172.21.15.72:0/4177164859 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00cbf20 con 0x7f03e00cb600 2023-12-12T09:52:40.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7fe755bc6640 1 -- 172.21.15.72:0/73303839 >> v2:172.21.15.72:3300/0 conn(0x7fe7500712a0 msgr2=0x7fe750071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7fe755bc6640 1 --2- 172.21.15.72:0/73303839 >> v2:172.21.15.72:3300/0 conn(0x7fe7500712a0 0x7fe750071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7fe755bc6640 1 -- 172.21.15.72:0/73303839 >> v1:172.21.15.72:6789/0 conn(0x7fe750143730 legacy=0x7fe75015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7fe755bc6640 1 -- 172.21.15.72:0/73303839 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750070560 con 0x7fe75015abc0 2023-12-12T09:52:40.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.411+0000 7f03e5c6c640 1 -- 172.21.15.72:0/4177164859 >> v2:172.21.15.72:3300/0 conn(0x7f03e006d230 msgr2=0x7f03e006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.411+0000 7f03e5c6c640 1 --2- 172.21.15.72:0/4177164859 >> v2:172.21.15.72:3300/0 conn(0x7f03e006d230 0x7f03e006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.411+0000 7f03e5c6c640 1 -- 172.21.15.72:0/4177164859 >> v1:172.21.15.72:6789/0 conn(0x7f03e006c010 legacy=0x7f03e006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.410+0000 7fae51153640 1 Processor -- start 2023-12-12T09:52:40.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.412+0000 7fae51153640 1 -- start start 2023-12-12T09:52:40.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.412+0000 7f03e5c6c640 1 -- 172.21.15.72:0/4177164859 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e015dda0 con 0x7f03e00cb600 2023-12-12T09:52:40.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.412+0000 7fe755bc6640 1 --2- 172.21.15.72:0/73303839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75015abc0 0x7fe750070d60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe744009870 tx=0x7fe74402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72a6579a42eb2002 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.413+0000 7fe7553c5640 1 -- 172.21.15.72:0/73303839 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744030840 con 0x7fe75015abc0 2023-12-12T09:52:40.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.413+0000 7f03e5c6c640 1 --2- 172.21.15.72:0/4177164859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e00cb600 0x7f03e00cb9e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f03d400b1e0 tx=0x7f03d402c180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c1d6dc82fb1c123 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.414+0000 7fe7553c5640 1 -- 172.21.15.72:0/73303839 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7440309a0 con 0x7fe75015abc0 2023-12-12T09:52:40.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.414+0000 7fe7553c5640 1 -- 172.21.15.72:0/73303839 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744030d30 con 0x7fe75015abc0 2023-12-12T09:52:40.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.414+0000 7fae51153640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34003970 0x7fae440d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.414+0000 7fae51153640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440537a0 0x7fae440e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.414+0000 7fae51153640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae440dcb00 0x7fae440dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fae51153640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440d5960 con 0x7fae34003970 2023-12-12T09:52:40.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fae51153640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440dd4f0 con 0x7fae440dcb00 2023-12-12T09:52:40.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fae4a59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440537a0 0x7fae440e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fae4a59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440537a0 0x7fae440e1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49480/0 (socket says 172.21.15.72:49480) 2023-12-12T09:52:40.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fe75862b640 1 -- 172.21.15.72:0/73303839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75015abc0 msgr2=0x7fe750070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fe75862b640 1 --2- 172.21.15.72:0/73303839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75015abc0 0x7fe750070d60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe744009870 tx=0x7fe74402b980 comp rx=0 tx=0).stop 2023-12-12T09:52:40.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fae4a59c640 1 -- 172.21.15.72:0/3252242243 learned_addr learned my addr 172.21.15.72:0/3252242243 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fae51153640 1 -- 172.21.15.72:0/3252242243 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440d5170 con 0x7fae440537a0 2023-12-12T09:52:40.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fe75862b640 1 -- 172.21.15.72:0/73303839 shutdown_connections 2023-12-12T09:52:40.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.415+0000 7fe75862b640 1 --2- 172.21.15.72:0/73303839 >> v2:172.21.15.72:3300/0 conn(0x7fe7500712a0 0x7fe750071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.416+0000 7fe75862b640 1 --2- 172.21.15.72:0/73303839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe75015abc0 0x7fe750070d60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.416+0000 7f03e4c6a640 1 -- 172.21.15.72:0/4177164859 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4034440 con 0x7f03e00cb600 2023-12-12T09:52:40.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.416+0000 7f03e4c6a640 1 -- 172.21.15.72:0/4177164859 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d40345a0 con 0x7f03e00cb600 2023-12-12T09:52:40.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.416+0000 7f03e4c6a640 1 -- 172.21.15.72:0/4177164859 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4034930 con 0x7f03e00cb600 2023-12-12T09:52:40.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.416+0000 7fe75862b640 1 -- 172.21.15.72:0/73303839 >> 172.21.15.72:0/73303839 conn(0x7fe7500b0940 msgr2=0x7fe7500b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.416+0000 7fae4b59e640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae440dcb00 0x7fae440dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.416+0000 7fae4ad9d640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34003970 0x7fae440d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.416+0000 7fe75862b640 1 -- 172.21.15.72:0/73303839 shutdown_connections 2023-12-12T09:52:40.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.417+0000 7fe75862b640 1 -- 172.21.15.72:0/73303839 wait complete. 2023-12-12T09:52:40.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.417+0000 7fae4b59e640 1 -- 172.21.15.72:0/3252242243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440537a0 msgr2=0x7fae440e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.417+0000 7fae4b59e640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440537a0 0x7fae440e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.417+0000 7fae4b59e640 1 -- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34003970 msgr2=0x7fae440d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.417+0000 7fae4b59e640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34003970 0x7fae440d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.417+0000 7fae4b59e640 1 -- 172.21.15.72:0/3252242243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440d3600 con 0x7fae440dcb00 2023-12-12T09:52:40.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.418+0000 7f03e7ed0640 1 -- 172.21.15.72:0/4177164859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e00cb600 msgr2=0x7f03e00cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.418+0000 7f1b9ffff640 1 -- 172.21.15.72:0/3573018097 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ba00538e0 con 0x7f1bac03e810 2023-12-12T09:52:40.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.418+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/4177164859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e00cb600 0x7f03e00cb9e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f03d400b1e0 tx=0x7f03d402c180 comp rx=0 tx=0).stop 2023-12-12T09:52:40.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.418+0000 7f03e7ed0640 1 -- 172.21.15.72:0/4177164859 shutdown_connections 2023-12-12T09:52:40.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.418+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/4177164859 >> v2:172.21.15.72:3300/0 conn(0x7f03e006d230 0x7f03e006d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.418+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/4177164859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e00cb600 0x7f03e00cb9e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.418+0000 7f03e7ed0640 1 -- 172.21.15.72:0/4177164859 >> 172.21.15.72:0/4177164859 conn(0x7f03e00b0940 msgr2=0x7f03e00b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.418+0000 7fae4b59e640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae440dcb00 0x7fae440dcfb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae4403ac30 tx=0x7fae40010590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.419+0000 7f03e7ed0640 1 -- 172.21.15.72:0/4177164859 shutdown_connections 2023-12-12T09:52:40.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.419+0000 7fae337fe640 1 -- 172.21.15.72:0/3252242243 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4001e640 con 0x7fae440dcb00 2023-12-12T09:52:40.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.419+0000 7fae51153640 1 -- 172.21.15.72:0/3252242243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440d53a0 con 0x7fae440dcb00 2023-12-12T09:52:40.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.419+0000 7fae337fe640 1 -- 172.21.15.72:0/3252242243 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4001e7a0 con 0x7fae440dcb00 2023-12-12T09:52:40.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.419+0000 7fae51153640 1 -- 172.21.15.72:0/3252242243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae440d2470 con 0x7fae440dcb00 2023-12-12T09:52:40.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.420+0000 7fe75862b640 1 Processor -- start 2023-12-12T09:52:40.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.420+0000 7fe75862b640 1 -- start start 2023-12-12T09:52:40.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.421+0000 7fae337fe640 1 -- 172.21.15.72:0/3252242243 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4001eb50 con 0x7fae440dcb00 2023-12-12T09:52:40.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.421+0000 7fae317fa640 1 -- 172.21.15.72:0/3252242243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae24005250 con 0x7fae440dcb00 2023-12-12T09:52:40.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.421+0000 7f03e7ed0640 1 -- 172.21.15.72:0/4177164859 wait complete. 2023-12-12T09:52:40.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.422+0000 7fe75862b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7500712a0 0x7fe750150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.422+0000 7fe75862b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe750143730 0x7fe75006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.423+0000 7fe75862b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75015abc0 0x7fe7501336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.423+0000 7fe7563c7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7500712a0 0x7fe750150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.423+0000 7fe7563c7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7500712a0 0x7fe750150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49484/0 (socket says 172.21.15.72:49484) 2023-12-12T09:52:40.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.424+0000 7fe7563c7640 1 -- 172.21.15.72:0/2282471585 learned_addr learned my addr 172.21.15.72:0/2282471585 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.424+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75015bd60 con 0x7fe750143730 2023-12-12T09:52:40.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.424+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75015b730 con 0x7fe75015abc0 2023-12-12T09:52:40.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.424+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75006d100 con 0x7fe7500712a0 2023-12-12T09:52:40.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.424+0000 7fe755bc6640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe750143730 0x7fe75006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.424+0000 7f03e7ed0640 1 Processor -- start 2023-12-12T09:52:40.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.424+0000 7f03e7ed0640 1 -- start start 2023-12-12T09:52:40.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.423+0000 7fae337fe640 1 -- 172.21.15.72:0/3252242243 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae400273f0 con 0x7fae440dcb00 2023-12-12T09:52:40.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.424+0000 7fae337fe640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fae3806ae40 0x7fae3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.425+0000 7fae337fe640 1 -- 172.21.15.72:0/3252242243 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fae4008e1d0 con 0x7fae440dcb00 2023-12-12T09:52:40.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.425+0000 7fe755bc6640 1 -- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7500712a0 msgr2=0x7fe750150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fe755bc6640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7500712a0 0x7fe750150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fe755bc6640 1 -- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75015abc0 msgr2=0x7fe7501336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fe756bc8640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75015abc0 0x7fe7501336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fae4ad9d640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fae3806ae40 0x7fae3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fa244f83640 1 Processor -- start 2023-12-12T09:52:40.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7f03e7ed0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e006c010 0x7f03e009b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7f03e7ed0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03e006d230 0x7f03e01ee670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fe755bc6640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75015abc0 0x7fe7501336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fe755bc6640 1 -- 172.21.15.72:0/2282471585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440094d0 con 0x7fe750143730 2023-12-12T09:52:40.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fe7563c7640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7500712a0 0x7fe750150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:40.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fe756bc8640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75015abc0 0x7fe7501336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:40.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7f03e7ed0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03e00cb600 0x7f03e0070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7f03e7ed0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e01f2fe0 con 0x7f03e00cb600 2023-12-12T09:52:40.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7f03e7ed0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e01f23d0 con 0x7f03e006d230 2023-12-12T09:52:40.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7f03e7ed0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e01f26b0 con 0x7f03e006c010 2023-12-12T09:52:40.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7fa244f83640 1 -- start start 2023-12-12T09:52:40.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.426+0000 7f03e646d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03e00cb600 0x7f03e0070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e5c6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e006c010 0x7f03e009b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e646d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03e00cb600 0x7f03e0070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40466/0 (socket says 172.21.15.72:40466) 2023-12-12T09:52:40.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e646d640 1 -- 172.21.15.72:0/2176591543 learned_addr learned my addr 172.21.15.72:0/2176591543 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7fe755bc6640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe750143730 0x7fe75006ca00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe744033000 tx=0x7fe744004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e546b640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03e006d230 0x7f03e01ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e646d640 1 -- 172.21.15.72:0/2176591543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e006c010 msgr2=0x7f03e009b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e646d640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e006c010 0x7f03e009b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e646d640 1 -- 172.21.15.72:0/2176591543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03e006d230 msgr2=0x7f03e01ee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e646d640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03e006d230 0x7f03e01ee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.427+0000 7f03e646d640 1 -- 172.21.15.72:0/2176591543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4009d60 con 0x7f03e00cb600 2023-12-12T09:52:40.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7fe74effd640 1 -- 172.21.15.72:0/2282471585 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744039c40 con 0x7fe750143730 2023-12-12T09:52:40.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7fe74effd640 1 -- 172.21.15.72:0/2282471585 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe744039da0 con 0x7fe750143730 2023-12-12T09:52:40.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7fae4ad9d640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fae3806ae40 0x7fae3806d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae3400ba10 tx=0x7fae4c08d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7fe74effd640 1 -- 172.21.15.72:0/2282471585 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744039c40 con 0x7fe750143730 2023-12-12T09:52:40.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75006d3e0 con 0x7fe750143730 2023-12-12T09:52:40.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7f03e646d640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03e00cb600 0x7f03e0070180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c780 tx=0x7f03d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7f03d27fc640 1 -- 172.21.15.72:0/2176591543 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8003d80 con 0x7f03e00cb600 2023-12-12T09:52:40.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7f03d27fc640 1 -- 172.21.15.72:0/2176591543 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d800ce90 con 0x7f03e00cb600 2023-12-12T09:52:40.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.428+0000 7f03d27fc640 1 -- 172.21.15.72:0/2176591543 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d80052b0 con 0x7f03e00cb600 2023-12-12T09:52:40.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.429+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe750134340 con 0x7fe750143730 2023-12-12T09:52:40.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.429+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e01f2990 con 0x7f03e00cb600 2023-12-12T09:52:40.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.429+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e00706c0 con 0x7f03e00cb600 2023-12-12T09:52:40.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.430+0000 7fa244f83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa240070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.430+0000 7fa244f83640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa24015a580 0x7fa24006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.430+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e0067910 con 0x7f03e00cb600 2023-12-12T09:52:40.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.430+0000 7fa244f83640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa24006d1d0 0x7fa2400b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.430+0000 7fa244f83640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa240153ce0 con 0x7fa24006d1d0 2023-12-12T09:52:40.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.430+0000 7fa244f83640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24006d650 con 0x7fa24015a580 2023-12-12T09:52:40.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.430+0000 7fa244f83640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2400cca90 con 0x7fa240153900 2023-12-12T09:52:40.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.431+0000 7fa23ed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa24006d1d0 0x7fa2400b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.431+0000 7fa23ed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa24006d1d0 0x7fa2400b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40478/0 (socket says 172.21.15.72:40478) 2023-12-12T09:52:40.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.431+0000 7fa23ed9d640 1 -- 172.21.15.72:0/2589540503 learned_addr learned my addr 172.21.15.72:0/2589540503 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.431+0000 7fa235d9b640 1 --2- 172.21.15.72:0/2589540503 >> v2:172.21.15.113:3300/0 conn(0x7fa24015a580 0x7fa24006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.431+0000 7fa23e59c640 1 --2- 172.21.15.72:0/2589540503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa240070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.432+0000 7fe74effd640 1 -- 172.21.15.72:0/2282471585 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe744042390 con 0x7fe750143730 2023-12-12T09:52:40.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.432+0000 7f03d27fc640 1 -- 172.21.15.72:0/2176591543 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f03d8005410 con 0x7f03e00cb600 2023-12-12T09:52:40.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.432+0000 7fa23ed9d640 1 -- 172.21.15.72:0/2589540503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 msgr2=0x7fa240070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.432+0000 7fa23ed9d640 1 --2- 172.21.15.72:0/2589540503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa240070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.432+0000 7fa23ed9d640 1 -- 172.21.15.72:0/2589540503 >> v2:172.21.15.113:3300/0 conn(0x7fa24015a580 msgr2=0x7fa24006cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.432+0000 7fa23ed9d640 1 --2- 172.21.15.72:0/2589540503 >> v2:172.21.15.113:3300/0 conn(0x7fa24015a580 0x7fa24006cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.432+0000 7fa23ed9d640 1 -- 172.21.15.72:0/2589540503 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2400ccd70 con 0x7fa24006d1d0 2023-12-12T09:52:40.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.432+0000 7fa23e59c640 1 --2- 172.21.15.72:0/2589540503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa240070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:40.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.434+0000 7fe74effd640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe73c06aea0 0x7fe73c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.434+0000 7fa23ed9d640 1 --2- 172.21.15.72:0/2589540503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24006d1d0 0x7fa2400b43e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2400b1a80 tx=0x7fa238030130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85061e0f1528b154 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.434+0000 7fe74effd640 1 -- 172.21.15.72:0/2282471585 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe7440ab790 con 0x7fe750143730 2023-12-12T09:52:40.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.434+0000 7f03d27fc640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03c406ae90 0x7f03c406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.434+0000 7fe7563c7640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe73c06aea0 0x7fe73c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.434+0000 7f03d27fc640 1 -- 172.21.15.72:0/2176591543 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f03d808b860 con 0x7f03e00cb600 2023-12-12T09:52:40.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.435+0000 7f03e5c6c640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03c406ae90 0x7f03c406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.435+0000 7fe7563c7640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe73c06aea0 0x7fe73c06d360 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe740007fd0 tx=0x7fe740009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.435+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe750067910 con 0x7fe750143730 2023-12-12T09:52:40.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.436+0000 7f03e5c6c640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03c406ae90 0x7f03c406d350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f03d4036000 tx=0x7f03d4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.439+0000 7fae337fe640 1 -- 172.21.15.72:0/3252242243 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae40059030 con 0x7fae440dcb00 2023-12-12T09:52:40.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.441+0000 7fa23dd9b640 1 -- 172.21.15.72:0/2589540503 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23803a400 con 0x7fa24006d1d0 2023-12-12T09:52:40.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.441+0000 7f03d27fc640 1 -- 172.21.15.72:0/2176591543 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03d80566b0 con 0x7f03e00cb600 2023-12-12T09:52:40.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.441+0000 7f0b5ffff640 1 Processor -- start 2023-12-12T09:52:40.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.442+0000 7f0b5ffff640 1 -- start start 2023-12-12T09:52:40.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.444+0000 7f0b5ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.444+0000 7f0b5ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0b5803c070 0x7f0b58058020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.444+0000 7f0b5ffff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b58058560 con 0x7f0b58053c80 2023-12-12T09:52:40.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.444+0000 7f0b5ffff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b58007b40 con 0x7f0b5803c070 2023-12-12T09:52:40.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.444+0000 7f0b5ffff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580587e0 con 0x7f0b580e3130 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.445+0000 7f0b5effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.445+0000 7f0b5e7fc640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0b5803c070 0x7f0b58058020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.445+0000 7f0b5f7fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f0b58053c80 0x7f0b580540f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52440/0 (socket says 172.21.15.72:52440) 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.445+0000 7f0b5f7fe640 1 -- 172.21.15.72:0/562138495 learned_addr learned my addr 172.21.15.72:0/562138495 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.445+0000 7f0b5effd640 1 -- 172.21.15.72:0/562138495 >> v2:172.21.15.72:3300/0 conn(0x7f0b5803c070 msgr2=0x7f0b58058020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.445+0000 7f0b5effd640 1 --2- 172.21.15.72:0/562138495 >> v2:172.21.15.72:3300/0 conn(0x7f0b5803c070 0x7f0b58058020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.445+0000 7f0b5effd640 1 -- 172.21.15.72:0/562138495 >> v1:172.21.15.72:6789/0 conn(0x7f0b58053c80 legacy=0x7f0b580540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.445+0000 7f0b5effd640 1 -- 172.21.15.72:0/562138495 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58058ac0 con 0x7f0b580e3130 2023-12-12T09:52:40.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.446+0000 7fe74effd640 1 -- 172.21.15.72:0/2282471585 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe7440765f0 con 0x7fe750143730 2023-12-12T09:52:40.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.448+0000 7fa23dd9b640 1 -- 172.21.15.72:0/2589540503 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa238034070 con 0x7fa24006d1d0 2023-12-12T09:52:40.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.448+0000 7fa23dd9b640 1 -- 172.21.15.72:0/2589540503 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23803a7c0 con 0x7fa24006d1d0 2023-12-12T09:52:40.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7f0b5effd640 1 --2- 172.21.15.72:0/562138495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580e7750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0b500097a0 tx=0x7f0b5002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b147e2d387dc8eac server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7fa244f83640 1 -- 172.21.15.72:0/2589540503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24006d1d0 msgr2=0x7fa2400b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7fa244f83640 1 --2- 172.21.15.72:0/2589540503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24006d1d0 0x7fa2400b43e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2400b1a80 tx=0x7fa238030130 comp rx=0 tx=0).stop 2023-12-12T09:52:40.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7f0b5dffb640 1 -- 172.21.15.72:0/562138495 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5002d040 con 0x7f0b580e3130 2023-12-12T09:52:40.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7fa244f83640 1 -- 172.21.15.72:0/2589540503 shutdown_connections 2023-12-12T09:52:40.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7fa244f83640 1 --2- 172.21.15.72:0/2589540503 >> v2:172.21.15.113:3300/0 conn(0x7fa24015a580 0x7fa24006cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7fa244f83640 1 --2- 172.21.15.72:0/2589540503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24006d1d0 0x7fa2400b43e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7fa244f83640 1 --2- 172.21.15.72:0/2589540503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa240070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7fa244f83640 1 -- 172.21.15.72:0/2589540503 >> 172.21.15.72:0/2589540503 conn(0x7fa2400b0a90 msgr2=0x7fa2400b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7f0b5dffb640 1 -- 172.21.15.72:0/562138495 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b50031810 con 0x7f0b580e3130 2023-12-12T09:52:40.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.449+0000 7f0b5dffb640 1 -- 172.21.15.72:0/562138495 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50031ba0 con 0x7f0b580e3130 2023-12-12T09:52:40.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.450+0000 7f0b5ffff640 1 -- 172.21.15.72:0/562138495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 msgr2=0x7f0b580e7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.450+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/562138495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580e7750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0b500097a0 tx=0x7f0b5002b8b0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.450+0000 7fa244f83640 1 -- 172.21.15.72:0/2589540503 shutdown_connections 2023-12-12T09:52:40.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.450+0000 7fa244f83640 1 -- 172.21.15.72:0/2589540503 wait complete. 2023-12-12T09:52:40.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.453+0000 7fa244f83640 1 Processor -- start 2023-12-12T09:52:40.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.453+0000 7fa244f83640 1 -- start start 2023-12-12T09:52:40.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.454+0000 7f0b5ffff640 1 -- 172.21.15.72:0/562138495 shutdown_connections 2023-12-12T09:52:40.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.454+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/562138495 >> v2:172.21.15.72:3300/0 conn(0x7f0b5803c070 0x7f0b58058020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.454+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/562138495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580e7750 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.454+0000 7f0b5ffff640 1 -- 172.21.15.72:0/562138495 >> 172.21.15.72:0/562138495 conn(0x7f0b58058eb0 msgr2=0x7f0b58059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.455+0000 7f0b5ffff640 1 -- 172.21.15.72:0/562138495 shutdown_connections 2023-12-12T09:52:40.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.455+0000 7f0b5ffff640 1 -- 172.21.15.72:0/562138495 wait complete. 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.455+0000 7fa244f83640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa24006d1d0 0x7fa24014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.456+0000 7fa244f83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa24015b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.456+0000 7fa244f83640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24015a580 0x7fa24015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.456+0000 7fa23e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa24006d1d0 0x7fa24014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.456+0000 7fa23e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa24006d1d0 0x7fa24014f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57842/0 (socket says 172.21.15.72:57842) 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.456+0000 7fa23e59c640 1 -- 172.21.15.72:0/2621773537 learned_addr learned my addr 172.21.15.72:0/2621773537 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.457+0000 7fa235d9b640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa24015b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.457+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24014c730 con 0x7fa24015a580 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.457+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2400b1d20 con 0x7fa24006d1d0 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.457+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2400b2000 con 0x7fa240153900 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.457+0000 7fa23ed9d640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24015a580 0x7fa24015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.459+0000 7fa235d9b640 1 -- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa24006d1d0 msgr2=0x7fa24014f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.459+0000 7fa235d9b640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa24006d1d0 0x7fa24014f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.459+0000 7fa235d9b640 1 -- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24015a580 msgr2=0x7fa24015bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.459+0000 7fa235d9b640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24015a580 0x7fa24015bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.459+0000 7fa235d9b640 1 -- 172.21.15.72:0/2621773537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa238008550 con 0x7fa240153900 2023-12-12T09:52:40.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.459+0000 7fa23ed9d640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24015a580 0x7fa24015bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:40.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.460+0000 7fa23e59c640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa24006d1d0 0x7fa24014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:40.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.461+0000 7f0b5ffff640 1 Processor -- start 2023-12-12T09:52:40.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.461+0000 7fa235d9b640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa24015b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa23000d520 tx=0x7fa23000d9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.461+0000 7f0b5ffff640 1 -- start start 2023-12-12T09:52:40.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.462+0000 7f0b5ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b5803c070 0x7f0b580da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.462+0000 7f0b5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b58053c80 0x7f0b580e4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.462+0000 7f0b5ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580df530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.462+0000 7f0b5ffff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58050f30 con 0x7f0b5803c070 2023-12-12T09:52:40.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.463+0000 7f0b5ffff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58050320 con 0x7f0b58053c80 2023-12-12T09:52:40.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.463+0000 7f0b5ffff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58050600 con 0x7f0b580e3130 2023-12-12T09:52:40.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.464+0000 7f0b5e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b58053c80 0x7f0b580e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.464+0000 7f0b5e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b58053c80 0x7f0b580e4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57844/0 (socket says 172.21.15.72:57844) 2023-12-12T09:52:40.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.464+0000 7fa2377fe640 1 -- 172.21.15.72:0/2621773537 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2300125f0 con 0x7fa240153900 2023-12-12T09:52:40.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.464+0000 7f0b5f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580df530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.465+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2400b22e0 con 0x7fa240153900 2023-12-12T09:52:40.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.464+0000 7f0b5f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580df530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49534/0 (socket says 172.21.15.72:49534) 2023-12-12T09:52:40.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.464+0000 7f0b5f7fe640 1 -- 172.21.15.72:0/3267269132 learned_addr learned my addr 172.21.15.72:0/3267269132 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.465+0000 7fa2377fe640 1 -- 172.21.15.72:0/2621773537 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa230012750 con 0x7fa240153900 2023-12-12T09:52:40.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.465+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24014f6a0 con 0x7fa240153900 2023-12-12T09:52:40.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.465+0000 7fa2377fe640 1 -- 172.21.15.72:0/2621773537 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230012ae0 con 0x7fa240153900 2023-12-12T09:52:40.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.465+0000 7f0b5f7fe640 1 -- 172.21.15.72:0/3267269132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b58053c80 msgr2=0x7f0b580e4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.466+0000 7f0b5f7fe640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b58053c80 0x7f0b580e4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.466+0000 7f0b5effd640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b5803c070 0x7f0b580da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.466+0000 7f0b5f7fe640 1 -- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b5803c070 msgr2=0x7f0b580da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.466+0000 7f0b5f7fe640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b5803c070 0x7f0b580da9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.466+0000 7f0b5f7fe640 1 -- 172.21.15.72:0/3267269132 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b50009400 con 0x7f0b580e3130 2023-12-12T09:52:40.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.466+0000 7f0b5e7fc640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b58053c80 0x7f0b580e4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:40.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.467+0000 7f0b5effd640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b5803c070 0x7f0b580da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:40.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.468+0000 7f0b5f7fe640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580df530 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0b600620a0 tx=0x7f0b60067ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.468+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa218005250 con 0x7fa240153900 2023-12-12T09:52:40.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.469+0000 7f0b477fe640 1 -- 172.21.15.72:0/3267269132 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b600730e0 con 0x7f0b580e3130 2023-12-12T09:52:40.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.469+0000 7f0b477fe640 1 -- 172.21.15.72:0/3267269132 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6008e070 con 0x7f0b580e3130 2023-12-12T09:52:40.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.469+0000 7f0b477fe640 1 -- 172.21.15.72:0/3267269132 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60073540 con 0x7f0b580e3130 2023-12-12T09:52:40.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.469+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b580508e0 con 0x7f0b580e3130 2023-12-12T09:52:40.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.471+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b580e3b10 con 0x7f0b580e3130 2023-12-12T09:52:40.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.472+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b580510a0 con 0x7f0b580e3130 2023-12-12T09:52:40.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.474+0000 7fa2377fe640 1 -- 172.21.15.72:0/2621773537 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa230012c40 con 0x7fa240153900 2023-12-12T09:52:40.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.476+0000 7fa2377fe640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa22406af10 0x7fa22406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.476+0000 7fa2377fe640 1 -- 172.21.15.72:0/2621773537 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa23008a580 con 0x7fa240153900 2023-12-12T09:52:40.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.476+0000 7fa23e59c640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa22406af10 0x7fa22406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.477+0000 7fa23e59c640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa22406af10 0x7fa22406d3d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa22800e6f0 tx=0x7fa2280078d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.487+0000 7fa2377fe640 1 -- 172.21.15.72:0/2621773537 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa230055460 con 0x7fa240153900 2023-12-12T09:52:40.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.488+0000 7f0b477fe640 1 -- 172.21.15.72:0/3267269132 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b6006a460 con 0x7f0b580e3130 2023-12-12T09:52:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.490+0000 7f0b477fe640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0b4c06ae40 0x7f0b4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.490+0000 7f0b477fe640 1 -- 172.21.15.72:0/3267269132 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0b60103880 con 0x7f0b580e3130 2023-12-12T09:52:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.491+0000 7f0b5effd640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0b4c06ae40 0x7f0b4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.492+0000 7f0b5effd640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0b4c06ae40 0x7f0b4c06d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b50036000 tx=0x7f0b5003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.499+0000 7f0b477fe640 1 -- 172.21.15.72:0/3267269132 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b600ce6b0 con 0x7f0b580e3130 2023-12-12T09:52:40.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.500+0000 7f247be9b640 1 Processor -- start 2023-12-12T09:52:40.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.501+0000 7f247be9b640 1 -- start start 2023-12-12T09:52:40.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.501+0000 7f11d6743640 1 Processor -- start 2023-12-12T09:52:40.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.501+0000 7f11d6743640 1 -- start start 2023-12-12T09:52:40.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.503+0000 7f247be9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f247406c410 0x7f247415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.503+0000 7f247be9b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f247406ca40 con 0x7f2474157a60 2023-12-12T09:52:40.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.503+0000 7f247be9b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f247415e510 con 0x7f247415e0e0 2023-12-12T09:52:40.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.503+0000 7f247be9b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2474071cd0 con 0x7f247406c410 2023-12-12T09:52:40.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.503+0000 7f11d6743640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d00d0360 0x7f11d00adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.504+0000 7f2479c37640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f2474157a60 0x7f247406de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52450/0 (socket says 172.21.15.72:52450) 2023-12-12T09:52:40.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.504+0000 7f2479c37640 1 -- 172.21.15.72:0/190833745 learned_addr learned my addr 172.21.15.72:0/190833745 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.504+0000 7f11d6743640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f11d006bf80 0x7f11d006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.504+0000 7f11d6743640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d00b07f0 con 0x7f11d015d580 2023-12-12T09:52:40.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.504+0000 7f11d6743640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d015d960 con 0x7f11d006bf80 2023-12-12T09:52:40.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.504+0000 7f11d6743640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d006c8d0 con 0x7f11d00d0360 2023-12-12T09:52:40.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.504+0000 7f2479436640 1 --2- 172.21.15.72:0/190833745 >> v2:172.21.15.113:3300/0 conn(0x7f247406c410 0x7f247415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.504+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d00d0360 0x7f11d00adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.505+0000 7f11cffff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f11d015d580 0x7f11d00cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54912/0 (socket says 172.21.15.72:54912) 2023-12-12T09:52:40.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.505+0000 7f11cffff640 1 -- 172.21.15.72:0/3611062314 learned_addr learned my addr 172.21.15.72:0/3611062314 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:40.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.505+0000 7f11d4ce0640 1 --2- 172.21.15.72:0/3611062314 >> v2:172.21.15.72:3300/0 conn(0x7f11d006bf80 0x7f11d006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.505+0000 7f2479436640 1 -- 172.21.15.72:0/190833745 >> v1:172.21.15.113:6789/0 conn(0x7f247415e0e0 legacy=0x7f24740cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.505+0000 7f2479436640 1 -- 172.21.15.72:0/190833745 >> v1:172.21.15.72:6789/0 conn(0x7f2474157a60 legacy=0x7f247406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.505+0000 7f2479436640 1 -- 172.21.15.72:0/190833745 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2474070f10 con 0x7f247406c410 2023-12-12T09:52:40.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.506+0000 7f11cf7fe640 1 -- 172.21.15.72:0/3611062314 >> v2:172.21.15.72:3300/0 conn(0x7f11d006bf80 msgr2=0x7f11d006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:40.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.505+0000 7f2478c35640 1 -- 172.21.15.72:0/190833745 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863009135 0 0) 0x7f247406ca40 con 0x7f2474157a60 2023-12-12T09:52:40.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.506+0000 7f11cf7fe640 1 --2- 172.21.15.72:0/3611062314 >> v2:172.21.15.72:3300/0 conn(0x7f11d006bf80 0x7f11d006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.506+0000 7f11cf7fe640 1 -- 172.21.15.72:0/3611062314 >> v1:172.21.15.113:6789/0 conn(0x7f11d015d580 legacy=0x7f11d00cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.506+0000 7f11cf7fe640 1 -- 172.21.15.72:0/3611062314 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00b0a40 con 0x7f11d00d0360 2023-12-12T09:52:40.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.506+0000 7f11d4ce0640 1 --2- 172.21.15.72:0/3611062314 >> v2:172.21.15.72:3300/0 conn(0x7f11d006bf80 0x7f11d006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:40.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.508+0000 7f11cf7fe640 1 --2- 172.21.15.72:0/3611062314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d00d0360 0x7f11d00adc60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11c80097a0 tx=0x7f11c802b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20507553834c4915 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.509+0000 7f11ceffd640 1 -- 172.21.15.72:0/3611062314 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8030710 con 0x7f11d00d0360 2023-12-12T09:52:40.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.509+0000 7f11ceffd640 1 -- 172.21.15.72:0/3611062314 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c802e050 con 0x7f11d00d0360 2023-12-12T09:52:40.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.509+0000 7f11ceffd640 1 -- 172.21.15.72:0/3611062314 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8030a90 con 0x7f11d00d0360 2023-12-12T09:52:40.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.509+0000 7f2479436640 1 --2- 172.21.15.72:0/190833745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247406c410 0x7f247415dba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f246c009d90 tx=0x7f246c02bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ac18e1d47ac8a20 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.510+0000 7f2478c35640 1 -- 172.21.15.72:0/190833745 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c02ca50 con 0x7f247406c410 2023-12-12T09:52:40.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.510+0000 7f2478c35640 1 -- 172.21.15.72:0/190833745 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246c02cbb0 con 0x7f247406c410 2023-12-12T09:52:40.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.510+0000 7f11d6743640 1 -- 172.21.15.72:0/3611062314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d00d0360 msgr2=0x7f11d00adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.510+0000 7f11d6743640 1 --2- 172.21.15.72:0/3611062314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d00d0360 0x7f11d00adc60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11c80097a0 tx=0x7f11c802b710 comp rx=0 tx=0).stop 2023-12-12T09:52:40.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.511+0000 7f11d6743640 1 -- 172.21.15.72:0/3611062314 shutdown_connections 2023-12-12T09:52:40.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.511+0000 7f11d6743640 1 --2- 172.21.15.72:0/3611062314 >> v2:172.21.15.72:3300/0 conn(0x7f11d006bf80 0x7f11d006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.511+0000 7f11d6743640 1 --2- 172.21.15.72:0/3611062314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d00d0360 0x7f11d00adc60 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11c80097a0 tx=0x7f11c802b710 comp rx=0 tx=0).stop 2023-12-12T09:52:40.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.511+0000 7f11d6743640 1 -- 172.21.15.72:0/3611062314 >> 172.21.15.72:0/3611062314 conn(0x7f11d00d0e90 msgr2=0x7f11d00d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.510+0000 7f2478c35640 1 -- 172.21.15.72:0/190833745 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c034440 con 0x7f247406c410 2023-12-12T09:52:40.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.511+0000 7f11d6743640 1 -- 172.21.15.72:0/3611062314 shutdown_connections 2023-12-12T09:52:40.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.511+0000 7f247be9b640 1 -- 172.21.15.72:0/190833745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247406c410 msgr2=0x7f247415dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.511+0000 7f247be9b640 1 --2- 172.21.15.72:0/190833745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247406c410 0x7f247415dba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f246c009d90 tx=0x7f246c02bd20 comp rx=0 tx=0).stop 2023-12-12T09:52:40.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.512+0000 7f247be9b640 1 -- 172.21.15.72:0/190833745 shutdown_connections 2023-12-12T09:52:40.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.512+0000 7f247be9b640 1 --2- 172.21.15.72:0/190833745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247406c410 0x7f247415dba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.512+0000 7f247be9b640 1 -- 172.21.15.72:0/190833745 >> 172.21.15.72:0/190833745 conn(0x7f24740b0a90 msgr2=0x7f24740b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.511+0000 7f11d6743640 1 -- 172.21.15.72:0/3611062314 wait complete. 2023-12-12T09:52:40.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.513+0000 7f247be9b640 1 -- 172.21.15.72:0/190833745 shutdown_connections 2023-12-12T09:52:40.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.513+0000 7f247be9b640 1 -- 172.21.15.72:0/190833745 wait complete. 2023-12-12T09:52:40.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.514+0000 7f11d6743640 1 Processor -- start 2023-12-12T09:52:40.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.515+0000 7f11d6743640 1 -- start start 2023-12-12T09:52:40.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.516+0000 7f247be9b640 1 Processor -- start 2023-12-12T09:52:40.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.516+0000 7f247be9b640 1 -- start start 2023-12-12T09:52:40.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.517+0000 7f11d6743640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d006bf80 0x7f11d00c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.517+0000 7f11d6743640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d015d580 0x7f11d0203c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.517+0000 7f11d6743640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11d00c4f00 0x7f11d02041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.518+0000 7f11d6743640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00c7790 con 0x7f11d00c4f00 2023-12-12T09:52:40.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.518+0000 7f11d6743640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01511b0 con 0x7f11d015d580 2023-12-12T09:52:40.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.518+0000 7f11cffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d006bf80 0x7f11d00c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.518+0000 7f11cffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d006bf80 0x7f11d00c49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49558/0 (socket says 172.21.15.72:49558) 2023-12-12T09:52:40.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.518+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d015d580 0x7f11d0203c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.519+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d0151490 con 0x7f11d006bf80 2023-12-12T09:52:40.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.519+0000 7f11cffff640 1 -- 172.21.15.72:0/363028634 learned_addr learned my addr 172.21.15.72:0/363028634 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.519+0000 7f247be9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247406c410 0x7f247414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f247be9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2474157a60 0x7f247413bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.519+0000 7f11d4ce0640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11d00c4f00 0x7f11d02041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f247be9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f247415e0e0 0x7f247413c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f11cffff640 1 -- 172.21.15.72:0/363028634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d015d580 msgr2=0x7f11d0203c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f11cffff640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d015d580 0x7f11d0203c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f11cffff640 1 -- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11d00c4f00 msgr2=0x7f11d02041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f11cffff640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11d00c4f00 0x7f11d02041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f11cffff640 1 -- 172.21.15.72:0/363028634 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009a70 con 0x7f11d006bf80 2023-12-12T09:52:40.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f2479c37640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247406c410 0x7f247414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.520+0000 7f2479c37640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247406c410 0x7f247414d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49574/0 (socket says 172.21.15.72:49574) 2023-12-12T09:52:40.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.521+0000 7f247a438640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f247415e0e0 0x7f247413c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.521+0000 7f247a438640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f247415e0e0 0x7f247413c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40522/0 (socket says 172.21.15.72:40522) 2023-12-12T09:52:40.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.521+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247414d920 con 0x7f247415e0e0 2023-12-12T09:52:40.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.521+0000 7f2479c37640 1 -- 172.21.15.72:0/3336203089 learned_addr learned my addr 172.21.15.72:0/3336203089 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:40.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.521+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2474155410 con 0x7f2474157a60 2023-12-12T09:52:40.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.521+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24741556f0 con 0x7f247406c410 2023-12-12T09:52:40.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.521+0000 7f2479436640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2474157a60 0x7f247413bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.521+0000 7f11cffff640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d006bf80 0x7f11d00c49c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c780 tx=0x7f11b800cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f11ccff9640 1 -- 172.21.15.72:0/363028634 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80072d0 con 0x7f11d006bf80 2023-12-12T09:52:40.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f247a438640 1 -- 172.21.15.72:0/3336203089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247406c410 msgr2=0x7f247414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f247a438640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247406c410 0x7f247414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f247a438640 1 -- 172.21.15.72:0/3336203089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2474157a60 msgr2=0x7f247413bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f247a438640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2474157a60 0x7f247413bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f247a438640 1 -- 172.21.15.72:0/3336203089 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c009a70 con 0x7f247415e0e0 2023-12-12T09:52:40.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f2479c37640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247406c410 0x7f247414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:40.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c8009400 con 0x7f11d006bf80 2023-12-12T09:52:40.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.522+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d0150e10 con 0x7f11d006bf80 2023-12-12T09:52:40.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.523+0000 7f247a438640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f247415e0e0 0x7f247413c050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f246400c460 tx=0x7f246400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.523+0000 7f246a7fc640 1 -- 172.21.15.72:0/3336203089 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464005480 con 0x7f247415e0e0 2023-12-12T09:52:40.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.523+0000 7f246a7fc640 1 -- 172.21.15.72:0/3336203089 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24640035f0 con 0x7f247415e0e0 2023-12-12T09:52:40.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.523+0000 7f246a7fc640 1 -- 172.21.15.72:0/3336203089 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24640039e0 con 0x7f247415e0e0 2023-12-12T09:52:40.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.523+0000 7f11ccff9640 1 -- 172.21.15.72:0/363028634 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8007430 con 0x7f11d006bf80 2023-12-12T09:52:40.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.523+0000 7f11ccff9640 1 -- 172.21.15.72:0/363028634 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80047e0 con 0x7f11d006bf80 2023-12-12T09:52:40.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.523+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247415c990 con 0x7f247415e0e0 2023-12-12T09:52:40.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.523+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2474155050 con 0x7f247415e0e0 2023-12-12T09:52:40.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.528+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b4005250 con 0x7f11d006bf80 2023-12-12T09:52:40.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.529+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c005250 con 0x7f247415e0e0 2023-12-12T09:52:40.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.528+0000 7f11ccff9640 1 -- 172.21.15.72:0/363028634 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11b8004960 con 0x7f11d006bf80 2023-12-12T09:52:40.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.529+0000 7f11ccff9640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11b006ae40 0x7f11b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.530+0000 7f11ccff9640 1 -- 172.21.15.72:0/363028634 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f11b8088f90 con 0x7f11d006bf80 2023-12-12T09:52:40.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.530+0000 7f11cf7fe640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11b006ae40 0x7f11b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.529+0000 7f246a7fc640 1 -- 172.21.15.72:0/3336203089 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2464003b40 con 0x7f247415e0e0 2023-12-12T09:52:40.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.531+0000 7f246a7fc640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f245806ae40 0x7f245806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:40.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.531+0000 7f246a7fc640 1 -- 172.21.15.72:0/3336203089 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2464088e30 con 0x7f247415e0e0 2023-12-12T09:52:40.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.532+0000 7f2479c37640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f245806ae40 0x7f245806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:40.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.533+0000 7f11cf7fe640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11b006ae40 0x7f11b006d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f11c8034000 tx=0x7f11c8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.534+0000 7f2479c37640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f245806ae40 0x7f245806d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2470000f60 tx=0x7f2470007e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:40.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.541+0000 7f246a7fc640 1 -- 172.21.15.72:0/3336203089 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f246408c030 con 0x7f247415e0e0 2023-12-12T09:52:40.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.545+0000 7f11ccff9640 1 -- 172.21.15.72:0/363028634 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11b8018020 con 0x7f11d006bf80 2023-12-12T09:52:40.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.826+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe75015bea0 con 0x7fe750143730 2023-12-12T09:52:40.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.827+0000 7fe74effd640 1 -- 172.21.15.72:0/2282471585 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe75015bea0 con 0x7fe750143730 2023-12-12T09:52:40.832 INFO:teuthology.orchestra.run.smithi072.stdout:55834574856 2023-12-12T09:52:40.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe73c06aea0 msgr2=0x7fe73c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe73c06aea0 0x7fe73c06d360 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe740007fd0 tx=0x7fe740009040 comp rx=0 tx=0).stop 2023-12-12T09:52:40.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe750143730 msgr2=0x7fe75006ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe750143730 0x7fe75006ca00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe744033000 tx=0x7fe744004730 comp rx=0 tx=0).stop 2023-12-12T09:52:40.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 shutdown_connections 2023-12-12T09:52:40.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe73c06aea0 0x7fe73c06d360 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75015abc0 0x7fe7501336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe750143730 0x7fe75006ca00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 --2- 172.21.15.72:0/2282471585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7500712a0 0x7fe750150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.831+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 >> 172.21.15.72:0/2282471585 conn(0x7fe7500b0940 msgr2=0x7fe750151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.832+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 shutdown_connections 2023-12-12T09:52:40.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.832+0000 7fe75862b640 1 -- 172.21.15.72:0/2282471585 wait complete. 2023-12-12T09:52:40.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.837+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f03e0074450 con 0x7f03e00cb600 2023-12-12T09:52:40.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.838+0000 7f03d27fc640 1 -- 172.21.15.72:0/2176591543 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f03d8059cd0 con 0x7f03e00cb600 2023-12-12T09:52:40.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.840+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1bac0df110 con 0x7f1bac03e810 2023-12-12T09:52:40.842 INFO:teuthology.orchestra.run.smithi072.stdout:55834574856 2023-12-12T09:52:40.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.841+0000 7f1b9ffff640 1 -- 172.21.15.72:0/3573018097 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ba0017020 con 0x7f1bac03e810 2023-12-12T09:52:40.844 INFO:teuthology.orchestra.run.smithi072.stdout:55834574856 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.842+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03c406ae90 msgr2=0x7f03c406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.842+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03c406ae90 0x7f03c406d350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f03d4036000 tx=0x7f03d4030000 comp rx=0 tx=0).stop 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.842+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03e00cb600 msgr2=0x7f03e0070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.842+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03e00cb600 0x7f03e0070180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c780 tx=0x7f03d800cc40 comp rx=0 tx=0).stop 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.843+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 shutdown_connections 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.843+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03c406ae90 0x7f03c406d350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.843+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03e00cb600 0x7f03e0070180 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.843+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03e006d230 0x7f03e01ee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.843+0000 7f03e7ed0640 1 --2- 172.21.15.72:0/2176591543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03e006c010 0x7f03e009b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.843+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 >> 172.21.15.72:0/2176591543 conn(0x7f03e00b0940 msgr2=0x7f03e00cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.843+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 shutdown_connections 2023-12-12T09:52:40.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.844+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1b9806aed0 msgr2=0x7f1b9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.844+0000 7f03e7ed0640 1 -- 172.21.15.72:0/2176591543 wait complete. 2023-12-12T09:52:40.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.844+0000 7f1bba197640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1b9806aed0 0x7f1b9806d390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8035000 tx=0x7f1ba8030000 comp rx=0 tx=0).stop 2023-12-12T09:52:40.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.844+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 msgr2=0x7f1bac04f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.844+0000 7f1bba197640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac04f0b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000caf0 tx=0x7f1ba000cef0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.845+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 shutdown_connections 2023-12-12T09:52:40.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.845+0000 7f1bba197640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.845+0000 7f1bba197640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bac0e3090 0x7f1bac03d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.845+0000 7f1bba197640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bac054100 0x7f1bac03cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.845+0000 7f1bba197640 1 --2- 172.21.15.72:0/3573018097 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bac03e810 0x7f1bac04f0b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.845+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 >> 172.21.15.72:0/3573018097 conn(0x7f1bac0559b0 msgr2=0x7f1bac0d9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.845+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 shutdown_connections 2023-12-12T09:52:40.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.845+0000 7f1bba197640 1 -- 172.21.15.72:0/3573018097 wait complete. 2023-12-12T09:52:40.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.857+0000 7fae317fa640 1 -- 172.21.15.72:0/3252242243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae240057d0 con 0x7fae440dcb00 2023-12-12T09:52:40.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.858+0000 7fae337fe640 1 -- 172.21.15.72:0/3252242243 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae4005c650 con 0x7fae440dcb00 2023-12-12T09:52:40.860 INFO:teuthology.orchestra.run.smithi072.stdout:55834574856 2023-12-12T09:52:40.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2023-12-12T09:52:40.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.861+0000 7fae317fa640 1 -- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fae3806ae40 msgr2=0x7fae3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.861+0000 7fae317fa640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fae3806ae40 0x7fae3806d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae3400ba10 tx=0x7fae4c08d040 comp rx=0 tx=0).stop 2023-12-12T09:52:40.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.862+0000 7fae317fa640 1 -- 172.21.15.72:0/3252242243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae440dcb00 msgr2=0x7fae440dcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.862+0000 7fae317fa640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae440dcb00 0x7fae440dcfb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae4403ac30 tx=0x7fae40010590 comp rx=0 tx=0).stop 2023-12-12T09:52:40.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.863+0000 7fae317fa640 1 -- 172.21.15.72:0/3252242243 shutdown_connections 2023-12-12T09:52:40.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.863+0000 7fae317fa640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fae3806ae40 0x7fae3806d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.863+0000 7fae317fa640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae440dcb00 0x7fae440dcfb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.863+0000 7fae317fa640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae440537a0 0x7fae440e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.863+0000 7fae317fa640 1 --2- 172.21.15.72:0/3252242243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae34003970 0x7fae440d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.863+0000 7fae317fa640 1 -- 172.21.15.72:0/3252242243 >> 172.21.15.72:0/3252242243 conn(0x7fae44009fb0 msgr2=0x7fae440dc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.863+0000 7fae317fa640 1 -- 172.21.15.72:0/3252242243 shutdown_connections 2023-12-12T09:52:40.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.863+0000 7fae317fa640 1 -- 172.21.15.72:0/3252242243 wait complete. 2023-12-12T09:52:40.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-12-12T09:52:40.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2023-12-12T09:52:40.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-12-12T09:52:40.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.903+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f245c005500 con 0x7f247415e0e0 2023-12-12T09:52:40.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.903+0000 7f246a7fc640 1 -- 172.21.15.72:0/3336203089 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2464017090 con 0x7f247415e0e0 2023-12-12T09:52:40.905 INFO:teuthology.orchestra.run.smithi072.stdout:55834574857 2023-12-12T09:52:40.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f245806ae40 msgr2=0x7f245806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f245806ae40 0x7f245806d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2470000f60 tx=0x7f2470007e10 comp rx=0 tx=0).stop 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f247415e0e0 msgr2=0x7f247413c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f247415e0e0 0x7f247413c050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f246400c460 tx=0x7f246400c920 comp rx=0 tx=0).stop 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 shutdown_connections 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f245806ae40 0x7f245806d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f247415e0e0 0x7f247413c050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2474157a60 0x7f247413bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 --2- 172.21.15.72:0/3336203089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247406c410 0x7f247414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.905+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 >> 172.21.15.72:0/3336203089 conn(0x7f24740b0a90 msgr2=0x7f2474158640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.906+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 shutdown_connections 2023-12-12T09:52:40.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.906+0000 7f247be9b640 1 -- 172.21.15.72:0/3336203089 wait complete. 2023-12-12T09:52:40.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.909+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f11b4005500 con 0x7f11d006bf80 2023-12-12T09:52:40.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.910+0000 7f11ccff9640 1 -- 172.21.15.72:0/363028634 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11b8053eb0 con 0x7f11d006bf80 2023-12-12T09:52:40.912 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-12-12T09:52:40.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.912+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11b006ae40 msgr2=0x7f11b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.912+0000 7f11d6743640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11b006ae40 0x7f11b006d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f11c8034000 tx=0x7f11c8008030 comp rx=0 tx=0).stop 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.912+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d006bf80 msgr2=0x7f11d00c49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.912+0000 7f11d6743640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d006bf80 0x7f11d00c49c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c780 tx=0x7f11b800cc40 comp rx=0 tx=0).stop 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.912+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 shutdown_connections 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.912+0000 7f11d6743640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11b006ae40 0x7f11b006d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.912+0000 7f11d6743640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11d00c4f00 0x7f11d02041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.913+0000 7f11d6743640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11d015d580 0x7f11d0203c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.913+0000 7f11d6743640 1 --2- 172.21.15.72:0/363028634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11d006bf80 0x7f11d00c49c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.913+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 >> 172.21.15.72:0/363028634 conn(0x7f11d00d0e90 msgr2=0x7f11d015acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.913+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 shutdown_connections 2023-12-12T09:52:40.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.913+0000 7f11d6743640 1 -- 172.21.15.72:0/363028634 wait complete. 2023-12-12T09:52:40.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.917+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa2180057d0 con 0x7fa240153900 2023-12-12T09:52:40.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.918+0000 7fa2377fe640 1 -- 172.21.15.72:0/2621773537 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa230058a80 con 0x7fa240153900 2023-12-12T09:52:40.920 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa22406af10 msgr2=0x7fa22406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa22406af10 0x7fa22406d3d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa22800e6f0 tx=0x7fa2280078d0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 msgr2=0x7fa24015b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa24015b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa23000d520 tx=0x7fa23000d9e0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 shutdown_connections 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa22406af10 0x7fa22406d3d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa24015a580 0x7fa24015bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa240153900 0x7fa24015b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 --2- 172.21.15.72:0/2621773537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa24006d1d0 0x7fa24014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 >> 172.21.15.72:0/2621773537 conn(0x7fa2400b0a90 msgr2=0x7fa240152100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 shutdown_connections 2023-12-12T09:52:40.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.921+0000 7fa244f83640 1 -- 172.21.15.72:0/2621773537 wait complete. 2023-12-12T09:52:40.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-12-12T09:52:40.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.925+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0b58050740 con 0x7f0b580e3130 2023-12-12T09:52:40.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.925+0000 7f0b477fe640 1 -- 172.21.15.72:0/3267269132 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b58050740 con 0x7f0b580e3130 2023-12-12T09:52:40.927 INFO:teuthology.orchestra.run.smithi072.stdout:55834574857 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0b4c06ae40 msgr2=0x7f0b4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0b4c06ae40 0x7f0b4c06d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b50036000 tx=0x7f0b5003c040 comp rx=0 tx=0).stop 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 msgr2=0x7f0b580df530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580df530 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0b600620a0 tx=0x7f0b60067ed0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 shutdown_connections 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0b4c06ae40 0x7f0b4c06d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b580e3130 0x7f0b580df530 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b58053c80 0x7f0b580e4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 --2- 172.21.15.72:0/3267269132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b5803c070 0x7f0b580da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:40.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 >> 172.21.15.72:0/3267269132 conn(0x7f0b58058eb0 msgr2=0x7f0b58057b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:40.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 shutdown_connections 2023-12-12T09:52:40.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:40.928+0000 7f0b5ffff640 1 -- 172.21.15.72:0/3267269132 wait complete. 2023-12-12T09:52:40.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2023-12-12T09:52:40.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2023-12-12T09:52:40.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2023-12-12T09:52:41.863 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-12T09:52:41.873 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-12T09:52:41.874 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-12T09:52:41.885 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-12T09:52:41.926 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-12T09:52:41.932 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-12T09:52:41.941 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-12T09:52:41.947 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.128+0000 7f58d359e640 1 Processor -- start 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.128+0000 7f58d359e640 1 -- start start 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.131+0000 7f58d359e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f58cc1051b0 0x7f58cc04f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.131+0000 7f58d359e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc054400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.131+0000 7f58d359e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58cc037bf0 0x7f58cc038040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.132+0000 7f58d359e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58cc050370 con 0x7f58cc1051b0 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.132+0000 7f58d359e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58cc054940 con 0x7f58cc037bf0 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.132+0000 7f58d359e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc04f7a0 con 0x7f58cc053fe0 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.132+0000 7f58d2d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58cc037bf0 0x7f58cc038040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.132+0000 7f58d2d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58cc037bf0 0x7f58cc038040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57896/0 (socket says 172.21.15.72:57896) 2023-12-12T09:52:42.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.132+0000 7f58d2d9d640 1 -- 172.21.15.72:0/237606638 learned_addr learned my addr 172.21.15.72:0/237606638 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.133+0000 7f58d1d9b640 1 --2- 172.21.15.72:0/237606638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc054400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.133+0000 7f58d259c640 1 --2- 172.21.15.72:0/237606638 >> v2:172.21.15.72:3300/0 conn(0x7f58cc1051b0 0x7f58cc04f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.133+0000 7f58d2d9d640 1 -- 172.21.15.72:0/237606638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 msgr2=0x7f58cc054400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.133+0000 7f58d2d9d640 1 --2- 172.21.15.72:0/237606638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc054400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.133+0000 7f58d2d9d640 1 -- 172.21.15.72:0/237606638 >> v2:172.21.15.72:3300/0 conn(0x7f58cc1051b0 msgr2=0x7f58cc04f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.133+0000 7f58d2d9d640 1 --2- 172.21.15.72:0/237606638 >> v2:172.21.15.72:3300/0 conn(0x7f58cc1051b0 0x7f58cc04f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.133+0000 7f58d2d9d640 1 -- 172.21.15.72:0/237606638 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc052fa0 con 0x7f58cc037bf0 2023-12-12T09:52:42.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.133+0000 7f58d259c640 1 --2- 172.21.15.72:0/237606638 >> v2:172.21.15.72:3300/0 conn(0x7f58cc1051b0 0x7f58cc04f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:42.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.135+0000 7f58d2d9d640 1 --2- 172.21.15.72:0/237606638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 0x7f58cc038040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f58cc04baf0 tx=0x7f58c802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3723f10754bf175 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.136+0000 7f58d159a640 1 -- 172.21.15.72:0/237606638 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8031830 con 0x7f58cc037bf0 2023-12-12T09:52:42.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.136+0000 7f58d159a640 1 -- 172.21.15.72:0/237606638 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8031990 con 0x7f58cc037bf0 2023-12-12T09:52:42.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.136+0000 7f58d159a640 1 -- 172.21.15.72:0/237606638 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8031d20 con 0x7f58cc037bf0 2023-12-12T09:52:42.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.137+0000 7f58d359e640 1 -- 172.21.15.72:0/237606638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 msgr2=0x7f58cc038040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.137+0000 7f58d359e640 1 --2- 172.21.15.72:0/237606638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 0x7f58cc038040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f58cc04baf0 tx=0x7f58c802b8d0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.137+0000 7f58d359e640 1 -- 172.21.15.72:0/237606638 shutdown_connections 2023-12-12T09:52:42.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.137+0000 7f58d359e640 1 --2- 172.21.15.72:0/237606638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 0x7f58cc038040 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.137+0000 7f58d359e640 1 --2- 172.21.15.72:0/237606638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc054400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.137+0000 7f58d359e640 1 --2- 172.21.15.72:0/237606638 >> v2:172.21.15.72:3300/0 conn(0x7f58cc1051b0 0x7f58cc04f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.137+0000 7f58d359e640 1 -- 172.21.15.72:0/237606638 >> 172.21.15.72:0/237606638 conn(0x7f58cc009f80 msgr2=0x7f58cc00a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.138+0000 7f58d359e640 1 -- 172.21.15.72:0/237606638 shutdown_connections 2023-12-12T09:52:42.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.138+0000 7f58d359e640 1 -- 172.21.15.72:0/237606638 wait complete. 2023-12-12T09:52:42.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.140+0000 7f58d359e640 1 Processor -- start 2023-12-12T09:52:42.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.141+0000 7f58d359e640 1 -- start start 2023-12-12T09:52:42.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.142+0000 7f58d359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 0x7f58cc04a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.142+0000 7f58d359e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc105fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.142+0000 7f58d359e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc1051b0 0x7f58cc006d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.143+0000 7f58d259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 0x7f58cc04a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.143+0000 7f58d1d9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc105fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.143+0000 7f58d1d9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc105fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49582/0 (socket says 172.21.15.72:49582) 2023-12-12T09:52:42.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.143+0000 7f58d259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 0x7f58cc04a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57898/0 (socket says 172.21.15.72:57898) 2023-12-12T09:52:42.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.143+0000 7f58d2d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc1051b0 0x7f58cc006d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.143+0000 7f58d2d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc1051b0 0x7f58cc006d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40528/0 (socket says 172.21.15.72:40528) 2023-12-12T09:52:42.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.144+0000 7f58d1d9b640 1 -- 172.21.15.72:0/1611596252 learned_addr learned my addr 172.21.15.72:0/1611596252 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.144+0000 7f58d359e640 1 -- 172.21.15.72:0/1611596252 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc107530 con 0x7f58cc1051b0 2023-12-12T09:52:42.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.145+0000 7f58d359e640 1 -- 172.21.15.72:0/1611596252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc04ac50 con 0x7f58cc037bf0 2023-12-12T09:52:42.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.145+0000 7f58d359e640 1 -- 172.21.15.72:0/1611596252 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc1068b0 con 0x7f58cc053fe0 2023-12-12T09:52:42.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.146+0000 7f58d2d9d640 1 -- 172.21.15.72:0/1611596252 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 msgr2=0x7f58cc105fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.146+0000 7f58d2d9d640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc105fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.146+0000 7f58d2d9d640 1 -- 172.21.15.72:0/1611596252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 msgr2=0x7f58cc04a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.146+0000 7f58d2d9d640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 0x7f58cc04a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.146+0000 7f58d2d9d640 1 -- 172.21.15.72:0/1611596252 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c80094d0 con 0x7f58cc1051b0 2023-12-12T09:52:42.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.147+0000 7f58d2d9d640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc1051b0 0x7f58cc006d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0509e0 tx=0x7f58c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.148+0000 7f58c2ffd640 1 -- 172.21.15.72:0/1611596252 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c803a820 con 0x7f58cc1051b0 2023-12-12T09:52:42.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.148+0000 7f58c2ffd640 1 -- 172.21.15.72:0/1611596252 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c803a980 con 0x7f58cc1051b0 2023-12-12T09:52:42.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.149+0000 7f58c2ffd640 1 -- 172.21.15.72:0/1611596252 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c803a820 con 0x7f58cc1051b0 2023-12-12T09:52:42.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.150+0000 7f58d359e640 1 -- 172.21.15.72:0/1611596252 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc106b90 con 0x7f58cc1051b0 2023-12-12T09:52:42.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.150+0000 7f58d359e640 1 -- 172.21.15.72:0/1611596252 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc107150 con 0x7f58cc1051b0 2023-12-12T09:52:42.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.152+0000 7f58c0ff9640 1 -- 172.21.15.72:0/1611596252 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac005250 con 0x7f58cc1051b0 2023-12-12T09:52:42.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.151+0000 7f58d1d9b640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:42.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.162+0000 7f58c2ffd640 1 -- 172.21.15.72:0/1611596252 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58c803aaf0 con 0x7f58cc1051b0 2023-12-12T09:52:42.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.163+0000 7f58c2ffd640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f58b406af00 0x7f58b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.167+0000 7f58d259c640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f58b406af00 0x7f58b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.168+0000 7f58d259c640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f58b406af00 0x7f58b406d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f58c4006fd0 tx=0x7f58c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.169+0000 7f58c2ffd640 1 -- 172.21.15.72:0/1611596252 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f58c807a080 con 0x7f58cc1051b0 2023-12-12T09:52:42.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.169+0000 7f58c2ffd640 1 -- 172.21.15.72:0/1611596252 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58c8076b50 con 0x7f58cc1051b0 2023-12-12T09:52:42.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.182+0000 7f7564e52640 1 Processor -- start 2023-12-12T09:52:42.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.182+0000 7f7564e52640 1 -- start start 2023-12-12T09:52:42.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.183+0000 7f5f8dd0f640 1 Processor -- start 2023-12-12T09:52:42.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.184+0000 7f5f8dd0f640 1 -- start start 2023-12-12T09:52:42.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.185+0000 7f7564e52640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f75580eb030 0x7f75580552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.185+0000 7f7564e52640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7558006170 0x7f7558006600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.185+0000 7f7564e52640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7558057e00 con 0x7f75580eb410 2023-12-12T09:52:42.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.185+0000 7f5f8dd0f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f881606d0 0x7f5f880d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.185+0000 7f5f8dd0f640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5f880b3e70 0x7f5f880cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f5f8dd0f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f880d1d30 con 0x7f5f880cfcd0 2023-12-12T09:52:42.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f5f8dd0f640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f880d0980 con 0x7f5f880b3e70 2023-12-12T09:52:42.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f5f8dd0f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f880d06a0 con 0x7f5f881606d0 2023-12-12T09:52:42.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f5f87fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5f880cfcd0 0x7f5f880cd9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54942/0 (socket says 172.21.15.72:54942) 2023-12-12T09:52:42.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f5f87fff640 1 -- 172.21.15.72:0/3578137888 learned_addr learned my addr 172.21.15.72:0/3578137888 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:42.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f5f877fe640 1 --2- 172.21.15.72:0/3578137888 >> v2:172.21.15.113:3300/0 conn(0x7f5f881606d0 0x7f5f880d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f5f86ffd640 1 --2- 172.21.15.72:0/3578137888 >> v2:172.21.15.72:3300/0 conn(0x7f5f880b3e70 0x7f5f880cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f5f877fe640 1 -- 172.21.15.72:0/3578137888 >> v2:172.21.15.72:3300/0 conn(0x7f5f880b3e70 msgr2=0x7f5f880cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f5f877fe640 1 --2- 172.21.15.72:0/3578137888 >> v2:172.21.15.72:3300/0 conn(0x7f5f880b3e70 0x7f5f880cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f5f877fe640 1 -- 172.21.15.72:0/3578137888 >> v1:172.21.15.113:6789/0 conn(0x7f5f880cfcd0 legacy=0x7f5f880cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f5f877fe640 1 -- 172.21.15.72:0/3578137888 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880d16e0 con 0x7f5f881606d0 2023-12-12T09:52:42.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.185+0000 7f7564e52640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580ebfd0 con 0x7f75580eb030 2023-12-12T09:52:42.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.185+0000 7f7564e52640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7558057970 con 0x7f7558006170 2023-12-12T09:52:42.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f755e59c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f75580eb030 0x7f75580552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f755e59c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f75580eb030 0x7f75580552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40542/0 (socket says 172.21.15.72:40542) 2023-12-12T09:52:42.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f755e59c640 1 -- 172.21.15.72:0/1222051011 learned_addr learned my addr 172.21.15.72:0/1222051011 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.186+0000 7f755ed9d640 1 --2- 172.21.15.72:0/1222051011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7558006170 0x7f7558006600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f755e59c640 1 -- 172.21.15.72:0/1222051011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7558006170 msgr2=0x7f7558006600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f755e59c640 1 --2- 172.21.15.72:0/1222051011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7558006170 0x7f7558006600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f755e59c640 1 -- 172.21.15.72:0/1222051011 >> v1:172.21.15.113:6789/0 conn(0x7f75580eb410 legacy=0x7f75580eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.187+0000 7f755e59c640 1 -- 172.21.15.72:0/1222051011 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580387b0 con 0x7f75580eb030 2023-12-12T09:52:42.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.188+0000 7f755e59c640 1 --2- 172.21.15.72:0/1222051011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb030 0x7f75580552d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7560068760 tx=0x7f7560066000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f52fda525ca7b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.189+0000 7f755d59a640 1 -- 172.21.15.72:0/1222051011 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600a6bb0 con 0x7f75580eb030 2023-12-12T09:52:42.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.189+0000 7f755d59a640 1 -- 172.21.15.72:0/1222051011 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75600a6d50 con 0x7f75580eb030 2023-12-12T09:52:42.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.189+0000 7f755d59a640 1 -- 172.21.15.72:0/1222051011 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600ae640 con 0x7f75580eb030 2023-12-12T09:52:42.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.189+0000 7f5f877fe640 1 --2- 172.21.15.72:0/3578137888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f881606d0 0x7f5f880d3a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f8806af70 tx=0x7f5f7c02b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e442e99638fcea2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.190+0000 7f5f867fc640 1 -- 172.21.15.72:0/3578137888 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c030710 con 0x7f5f881606d0 2023-12-12T09:52:42.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.190+0000 7f5f867fc640 1 -- 172.21.15.72:0/3578137888 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c02e050 con 0x7f5f881606d0 2023-12-12T09:52:42.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.190+0000 7f7564e52640 1 -- 172.21.15.72:0/1222051011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb030 msgr2=0x7f75580552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.190+0000 7f7564e52640 1 --2- 172.21.15.72:0/1222051011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb030 0x7f75580552d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7560068760 tx=0x7f7560066000 comp rx=0 tx=0).stop 2023-12-12T09:52:42.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.190+0000 7f5f867fc640 1 -- 172.21.15.72:0/3578137888 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c030a90 con 0x7f5f881606d0 2023-12-12T09:52:42.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f7564e52640 1 -- 172.21.15.72:0/1222051011 shutdown_connections 2023-12-12T09:52:42.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f7564e52640 1 --2- 172.21.15.72:0/1222051011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7558006170 0x7f7558006600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f7564e52640 1 --2- 172.21.15.72:0/1222051011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb030 0x7f75580552d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f7564e52640 1 -- 172.21.15.72:0/1222051011 >> 172.21.15.72:0/1222051011 conn(0x7f75580559b0 msgr2=0x7f7558055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f5f8dd0f640 1 -- 172.21.15.72:0/3578137888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f881606d0 msgr2=0x7f5f880d3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f5f8dd0f640 1 --2- 172.21.15.72:0/3578137888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f881606d0 0x7f5f880d3a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f8806af70 tx=0x7f5f7c02b710 comp rx=0 tx=0).stop 2023-12-12T09:52:42.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f5f8dd0f640 1 -- 172.21.15.72:0/3578137888 shutdown_connections 2023-12-12T09:52:42.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f7564e52640 1 -- 172.21.15.72:0/1222051011 shutdown_connections 2023-12-12T09:52:42.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f7564e52640 1 -- 172.21.15.72:0/1222051011 wait complete. 2023-12-12T09:52:42.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f5f8dd0f640 1 --2- 172.21.15.72:0/3578137888 >> v2:172.21.15.72:3300/0 conn(0x7f5f880b3e70 0x7f5f880cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f5f8dd0f640 1 --2- 172.21.15.72:0/3578137888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f881606d0 0x7f5f880d3a80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.191+0000 7f5f8dd0f640 1 -- 172.21.15.72:0/3578137888 >> 172.21.15.72:0/3578137888 conn(0x7f5f880d2f60 msgr2=0x7f5f880d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.192+0000 7f5f8dd0f640 1 -- 172.21.15.72:0/3578137888 shutdown_connections 2023-12-12T09:52:42.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.192+0000 7f5f8dd0f640 1 -- 172.21.15.72:0/3578137888 wait complete. 2023-12-12T09:52:42.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.194+0000 7fda6e01c640 1 Processor -- start 2023-12-12T09:52:42.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.194+0000 7f7564e52640 1 Processor -- start 2023-12-12T09:52:42.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.195+0000 7fda6e01c640 1 -- start start 2023-12-12T09:52:42.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.195+0000 7f7564e52640 1 -- start start 2023-12-12T09:52:42.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7f7564e52640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7558006170 0x7f75580e1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7fda6e01c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 0x7fda6806cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7f5f8dd0f640 1 Processor -- start 2023-12-12T09:52:42.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7f7564e52640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75580eb030 0x7f7558189520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7f5f8dd0f640 1 -- start start 2023-12-12T09:52:42.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7f7564e52640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb410 0x7f7558189a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7fda6e01c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fda680b3cc0 0x7fda6806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7fda6e01c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6806d9a0 con 0x7fda680700e0 2023-12-12T09:52:42.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7f755dd9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75580eb030 0x7f7558189520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7f755dd9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75580eb030 0x7f7558189520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49596/0 (socket says 172.21.15.72:49596) 2023-12-12T09:52:42.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7f755dd9b640 1 -- 172.21.15.72:0/2945894801 learned_addr learned my addr 172.21.15.72:0/2945894801 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7fda6e01c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda680cd760 con 0x7fda680b3cc0 2023-12-12T09:52:42.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755800bdb0 con 0x7f75580eb410 2023-12-12T09:52:42.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7f755e59c640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7558006170 0x7f75580e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.197+0000 7fda6e01c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda680ccb50 con 0x7fda6815d130 2023-12-12T09:52:42.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7fda5effd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fda680700e0 0x7fda680b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52462/0 (socket says 172.21.15.72:52462) 2023-12-12T09:52:42.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7fda5effd640 1 -- 172.21.15.72:0/3040639330 learned_addr learned my addr 172.21.15.72:0/3040639330 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:42.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755800c090 con 0x7f7558006170 2023-12-12T09:52:42.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755800c370 con 0x7f75580eb030 2023-12-12T09:52:42.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7f755ed9d640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb410 0x7f7558189a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.198+0000 7fda67fff640 1 --2- 172.21.15.72:0/3040639330 >> v2:172.21.15.72:3300/0 conn(0x7fda680b3cc0 0x7fda6806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.199+0000 7f5f8dd0f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f880b3e70 0x7f5f880cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.199+0000 7f5f8dd0f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f880cfcd0 0x7f5f880c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.199+0000 7fda677fe640 1 --2- 172.21.15.72:0/3040639330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 0x7fda6806cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.199+0000 7f5f8dd0f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f881606d0 0x7f5f881f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.199+0000 7f5f8dd0f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8815e890 con 0x7f5f880b3e70 2023-12-12T09:52:42.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.199+0000 7f5f8dd0f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880c8480 con 0x7f5f880cfcd0 2023-12-12T09:52:42.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.199+0000 7f5f8dd0f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b89b0 con 0x7f5f881606d0 2023-12-12T09:52:42.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f755e59c640 1 -- 172.21.15.72:0/2945894801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75580eb030 msgr2=0x7f7558189520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f755e59c640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75580eb030 0x7f7558189520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f755e59c640 1 -- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb410 msgr2=0x7f7558189a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f755e59c640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb410 0x7f7558189a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f755e59c640 1 -- 172.21.15.72:0/2945894801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560074230 con 0x7f7558006170 2023-12-12T09:52:42.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f5f877fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f880b3e70 0x7f5f880cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f5f877fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f880b3e70 0x7f5f880cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40570/0 (socket says 172.21.15.72:40570) 2023-12-12T09:52:42.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f5f877fe640 1 -- 172.21.15.72:0/304988225 learned_addr learned my addr 172.21.15.72:0/304988225 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7fda67fff640 1 -- 172.21.15.72:0/3040639330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 msgr2=0x7fda6806cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7fda67fff640 1 --2- 172.21.15.72:0/3040639330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 0x7fda6806cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7fda67fff640 1 -- 172.21.15.72:0/3040639330 >> v1:172.21.15.72:6789/0 conn(0x7fda680700e0 legacy=0x7fda680b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7fda67fff640 1 -- 172.21.15.72:0/3040639330 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda680cce30 con 0x7fda680b3cc0 2023-12-12T09:52:42.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f5f86ffd640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f880cfcd0 0x7f5f880c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.200+0000 7f5f87fff640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f881606d0 0x7f5f881f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f755e59c640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7558006170 0x7f75580e1100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75600697f0 tx=0x7f7560069ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f7556ffd640 1 -- 172.21.15.72:0/2945894801 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600ba650 con 0x7f7558006170 2023-12-12T09:52:42.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f5f877fe640 1 -- 172.21.15.72:0/304988225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f881606d0 msgr2=0x7f5f881f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f5f877fe640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f881606d0 0x7f5f881f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f5f877fe640 1 -- 172.21.15.72:0/304988225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f880cfcd0 msgr2=0x7f5f880c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f5f877fe640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f880cfcd0 0x7f5f880c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f5f877fe640 1 -- 172.21.15.72:0/304988225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c009400 con 0x7f5f880b3e70 2023-12-12T09:52:42.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755800c650 con 0x7f7558006170 2023-12-12T09:52:42.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.202+0000 7fda67fff640 1 --2- 172.21.15.72:0/3040639330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680b3cc0 0x7fda6806d060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda60009d90 tx=0x7fda6002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40dcfbcfd4f7fb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.202+0000 7f5f877fe640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f880b3e70 0x7f5f880cf2a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009890 tx=0x7f5f7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.202+0000 7f5f6ffff640 1 -- 172.21.15.72:0/304988225 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c03c8e0 con 0x7f5f880b3e70 2023-12-12T09:52:42.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.201+0000 7f7556ffd640 1 -- 172.21.15.72:0/2945894801 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75600ba7b0 con 0x7f7558006170 2023-12-12T09:52:42.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.202+0000 7f7556ffd640 1 -- 172.21.15.72:0/2945894801 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600baba0 con 0x7f7558006170 2023-12-12T09:52:42.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.202+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755818a300 con 0x7f7558006170 2023-12-12T09:52:42.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.203+0000 7f5f6ffff640 1 -- 172.21.15.72:0/304988225 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c03ca40 con 0x7f5f880b3e70 2023-12-12T09:52:42.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.203+0000 7f5f6ffff640 1 -- 172.21.15.72:0/304988225 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c03cdf0 con 0x7f5f880b3e70 2023-12-12T09:52:42.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.203+0000 7f5f8dd0f640 1 -- 172.21.15.72:0/304988225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f880b8c90 con 0x7f5f880b3e70 2023-12-12T09:52:42.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.203+0000 7fda66ffd640 1 -- 172.21.15.72:0/3040639330 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6002cc50 con 0x7fda680b3cc0 2023-12-12T09:52:42.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.203+0000 7f5f8dd0f640 1 -- 172.21.15.72:0/304988225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f880c89b0 con 0x7f5f880b3e70 2023-12-12T09:52:42.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.203+0000 7fda66ffd640 1 -- 172.21.15.72:0/3040639330 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6002cdb0 con 0x7fda680b3cc0 2023-12-12T09:52:42.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.203+0000 7fda66ffd640 1 -- 172.21.15.72:0/3040639330 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60034660 con 0x7fda680b3cc0 2023-12-12T09:52:42.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.204+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75580057b0 con 0x7f7558006170 2023-12-12T09:52:42.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.204+0000 7fda6e01c640 1 -- 172.21.15.72:0/3040639330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680b3cc0 msgr2=0x7fda6806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.204+0000 7fda6e01c640 1 --2- 172.21.15.72:0/3040639330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680b3cc0 0x7fda6806d060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda60009d90 tx=0x7fda6002bea0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.205+0000 7f5f6dffb640 1 -- 172.21.15.72:0/304988225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f60005250 con 0x7f5f880b3e70 2023-12-12T09:52:42.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.207+0000 7f7556ffd640 1 -- 172.21.15.72:0/2945894801 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f75600bad00 con 0x7f7558006170 2023-12-12T09:52:42.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.209+0000 7f7556ffd640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f754006af10 0x7f754006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.209+0000 7f7556ffd640 1 -- 172.21.15.72:0/2945894801 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7560126390 con 0x7f7558006170 2023-12-12T09:52:42.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.210+0000 7f755dd9b640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f754006af10 0x7f754006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.209+0000 7f5f6ffff640 1 -- 172.21.15.72:0/304988225 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f7c039070 con 0x7f5f880b3e70 2023-12-12T09:52:42.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.210+0000 7f5f6ffff640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5f7406ae70 0x7f5f7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.210+0000 7f5f6ffff640 1 -- 172.21.15.72:0/304988225 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5f7c0ab850 con 0x7f5f880b3e70 2023-12-12T09:52:42.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.211+0000 7f5f86ffd640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5f7406ae70 0x7f5f7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.212+0000 7fda6e01c640 1 -- 172.21.15.72:0/3040639330 shutdown_connections 2023-12-12T09:52:42.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.212+0000 7fda6e01c640 1 --2- 172.21.15.72:0/3040639330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680b3cc0 0x7fda6806d060 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.212+0000 7fda6e01c640 1 --2- 172.21.15.72:0/3040639330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 0x7fda6806cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.212+0000 7fda6e01c640 1 -- 172.21.15.72:0/3040639330 >> 172.21.15.72:0/3040639330 conn(0x7fda680b0940 msgr2=0x7fda680b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.212+0000 7f755dd9b640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f754006af10 0x7f754006d3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f755804eaf0 tx=0x7f7548007ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.212+0000 7fda6e01c640 1 -- 172.21.15.72:0/3040639330 shutdown_connections 2023-12-12T09:52:42.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.213+0000 7f5f86ffd640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5f7406ae70 0x7f5f7406d330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f880b5dd0 tx=0x7f5f78003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.213+0000 7fda6e01c640 1 -- 172.21.15.72:0/3040639330 wait complete. 2023-12-12T09:52:42.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.215+0000 7fda6e01c640 1 Processor -- start 2023-12-12T09:52:42.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.215+0000 7fda6e01c640 1 -- start start 2023-12-12T09:52:42.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.216+0000 7f5f6ffff640 1 -- 172.21.15.72:0/304988225 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f7c076810 con 0x7f5f880b3e70 2023-12-12T09:52:42.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.216+0000 7fda6e01c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680700e0 0x7fda681603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.217+0000 7fda6e01c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda680b3cc0 0x7fda68155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.217+0000 7fda6e01c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 0x7fda68155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.217+0000 7fda6e01c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6815e890 con 0x7fda680700e0 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.217+0000 7fda6e01c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda68154ce0 con 0x7fda680b3cc0 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.217+0000 7fda6e01c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6815e240 con 0x7fda6815d130 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.217+0000 7fda677fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680700e0 0x7fda681603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.217+0000 7fda677fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680700e0 0x7fda681603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40578/0 (socket says 172.21.15.72:40578) 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.217+0000 7fda677fe640 1 -- 172.21.15.72:0/2340559677 learned_addr learned my addr 172.21.15.72:0/2340559677 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.218+0000 7fda5effd640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda680b3cc0 0x7fda68155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.218+0000 7fda67fff640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 0x7fda68155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.218+0000 7fda677fe640 1 -- 172.21.15.72:0/2340559677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 msgr2=0x7fda68155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.218+0000 7fda677fe640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 0x7fda68155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.218+0000 7fda677fe640 1 -- 172.21.15.72:0/2340559677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda680b3cc0 msgr2=0x7fda68155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.218+0000 7fda677fe640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda680b3cc0 0x7fda68155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.218+0000 7fda677fe640 1 -- 172.21.15.72:0/2340559677 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda60009a70 con 0x7fda680700e0 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.218+0000 7fda677fe640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680700e0 0x7fda681603d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fda5800d5f0 tx=0x7fda5800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.219+0000 7fda64ff9640 1 -- 172.21.15.72:0/2340559677 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580125f0 con 0x7fda680700e0 2023-12-12T09:52:42.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.219+0000 7fda64ff9640 1 -- 172.21.15.72:0/2340559677 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda58012750 con 0x7fda680700e0 2023-12-12T09:52:42.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.219+0000 7fda64ff9640 1 -- 172.21.15.72:0/2340559677 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58012b00 con 0x7fda680700e0 2023-12-12T09:52:42.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.221+0000 7f7556ffd640 1 -- 172.21.15.72:0/2945894801 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75600f1300 con 0x7f7558006170 2023-12-12T09:52:42.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.220+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6815e520 con 0x7fda680700e0 2023-12-12T09:52:42.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.220+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda68152a80 con 0x7fda680700e0 2023-12-12T09:52:42.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.221+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda68067910 con 0x7fda680700e0 2023-12-12T09:52:42.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.225+0000 7fda64ff9640 1 -- 172.21.15.72:0/2340559677 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda58012c60 con 0x7fda680700e0 2023-12-12T09:52:42.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.226+0000 7fda64ff9640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fda4c06af10 0x7fda4c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.226+0000 7fda64ff9640 1 -- 172.21.15.72:0/2340559677 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fda5808a7b0 con 0x7fda680700e0 2023-12-12T09:52:42.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.230+0000 7fda5effd640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fda4c06af10 0x7fda4c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.230+0000 7fda64ff9640 1 -- 172.21.15.72:0/2340559677 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda58055600 con 0x7fda680700e0 2023-12-12T09:52:42.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.231+0000 7fda5effd640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fda4c06af10 0x7fda4c06d3d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda6815f360 tx=0x7fda50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.269+0000 7f554ffff640 1 Processor -- start 2023-12-12T09:52:42.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.269+0000 7f554ffff640 1 -- start start 2023-12-12T09:52:42.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.271+0000 7f554ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f555015bc00 0x7f555006c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.271+0000 7f554ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5550071ea0 0x7f55500ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.271+0000 7f554ffff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5550070070 con 0x7f555015b820 2023-12-12T09:52:42.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.271+0000 7f554ffff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555006c290 con 0x7f5550071ea0 2023-12-12T09:52:42.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.271+0000 7f554ffff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55500702c0 con 0x7f555015bc00 2023-12-12T09:52:42.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.271+0000 7fa5effff640 1 Processor -- start 2023-12-12T09:52:42.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.272+0000 7fa5effff640 1 -- start start 2023-12-12T09:52:42.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.272+0000 7f554f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5550071ea0 0x7f55500ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.273+0000 7f554f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5550071ea0 0x7f55500ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57936/0 (socket says 172.21.15.72:57936) 2023-12-12T09:52:42.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.273+0000 7f554f7fe640 1 -- 172.21.15.72:0/1269045951 learned_addr learned my addr 172.21.15.72:0/1269045951 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.272+0000 7f554effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f555015b820 0x7f55501623c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54954/0 (socket says 172.21.15.72:54954) 2023-12-12T09:52:42.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.272+0000 7f554e7fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f555015bc00 0x7f555006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.273+0000 7f554f7fe640 1 -- 172.21.15.72:0/1269045951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f555015bc00 msgr2=0x7f555006c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.273+0000 7f554f7fe640 1 --2- 172.21.15.72:0/1269045951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f555015bc00 0x7f555006c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.273+0000 7f554f7fe640 1 -- 172.21.15.72:0/1269045951 >> v1:172.21.15.113:6789/0 conn(0x7f555015b820 legacy=0x7f55501623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.273+0000 7f554f7fe640 1 -- 172.21.15.72:0/1269045951 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55500705a0 con 0x7f5550071ea0 2023-12-12T09:52:42.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.274+0000 7fa5effff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa5f00d0d70 0x7fa5f00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.274+0000 7fa5effff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f00b2280 con 0x7fa5f00ce5b0 2023-12-12T09:52:42.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.274+0000 7fa5effff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f00cee70 con 0x7fa5f00d13c0 2023-12-12T09:52:42.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.274+0000 7fa5effff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f00d11c0 con 0x7fa5f00d0d70 2023-12-12T09:52:42.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.274+0000 7fa5eeffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa5f00d13c0 0x7fa5f00d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:54968/0 (socket says 172.21.15.72:54968) 2023-12-12T09:52:42.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.274+0000 7fa5eeffd640 1 -- 172.21.15.72:0/2853879556 learned_addr learned my addr 172.21.15.72:0/2853879556 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:42.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.275+0000 7fa5ef7fe640 1 --2- 172.21.15.72:0/2853879556 >> v2:172.21.15.72:3300/0 conn(0x7fa5f00d0d70 0x7fa5f00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.275+0000 7efedffff640 1 Processor -- start 2023-12-12T09:52:42.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.275+0000 7f554f7fe640 1 --2- 172.21.15.72:0/1269045951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5550071ea0 0x7f55500ceef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5540009870 tx=0x7f554002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbeba3e72700497e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.276+0000 7fa5edffb640 1 -- 172.21.15.72:0/2853879556 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057674588 0 0) 0x7fa5f00cee70 con 0x7fa5f00d13c0 2023-12-12T09:52:42.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.276+0000 7fa5edffb640 1 -- 172.21.15.72:0/2853879556 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d8003520 con 0x7fa5f00d13c0 2023-12-12T09:52:42.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.276+0000 7fa5ef7fe640 1 -- 172.21.15.72:0/2853879556 >> v1:172.21.15.113:6789/0 conn(0x7fa5f00d13c0 legacy=0x7fa5f00d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.276+0000 7fa5ef7fe640 1 -- 172.21.15.72:0/2853879556 >> v1:172.21.15.72:6789/0 conn(0x7fa5f00ce5b0 legacy=0x7fa5f00b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.276+0000 7fa5ef7fe640 1 -- 172.21.15.72:0/2853879556 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f00ceb90 con 0x7fa5f00d0d70 2023-12-12T09:52:42.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.277+0000 7f554dffb640 1 -- 172.21.15.72:0/1269045951 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540031640 con 0x7f5550071ea0 2023-12-12T09:52:42.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.277+0000 7f554dffb640 1 -- 172.21.15.72:0/1269045951 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55400317a0 con 0x7f5550071ea0 2023-12-12T09:52:42.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.277+0000 7f554dffb640 1 -- 172.21.15.72:0/1269045951 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540031b30 con 0x7f5550071ea0 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.276+0000 7efedffff640 1 -- start start 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.277+0000 7efedffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efed80362c0 0x7efed800dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed8056750 0x7efed800e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efed8049bd0 0x7efed80eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedffff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed8055920 con 0x7efed80362c0 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedffff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed8055490 con 0x7efed8049bd0 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedffff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed8054880 con 0x7efed8056750 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedeffd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efed80362c0 0x7efed800dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedeffd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efed80362c0 0x7efed800dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40600/0 (socket says 172.21.15.72:40600) 2023-12-12T09:52:42.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7fa5ef7fe640 1 --2- 172.21.15.72:0/2853879556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 0x7fa5f00b0e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009d90 tx=0x7fa5e802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2dbda679476f52 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedeffd640 1 -- 172.21.15.72:0/1127280139 learned_addr learned my addr 172.21.15.72:0/1127280139 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efede7fc640 1 --2- 172.21.15.72:0/1127280139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed8056750 0x7efed800e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.278+0000 7efedf7fe640 1 --2- 172.21.15.72:0/1127280139 >> v2:172.21.15.113:3300/0 conn(0x7efed8049bd0 0x7efed80eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7efedeffd640 1 -- 172.21.15.72:0/1127280139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed8056750 msgr2=0x7efed800e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7efedeffd640 1 --2- 172.21.15.72:0/1127280139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed8056750 0x7efed800e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7efedeffd640 1 -- 172.21.15.72:0/1127280139 >> v2:172.21.15.113:3300/0 conn(0x7efed8049bd0 msgr2=0x7efed80eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7fa5edffb640 1 -- 172.21.15.72:0/2853879556 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e802cc70 con 0x7fa5f00d0d70 2023-12-12T09:52:42.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7f554ffff640 1 -- 172.21.15.72:0/1269045951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5550071ea0 msgr2=0x7f55500ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7f554ffff640 1 --2- 172.21.15.72:0/1269045951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5550071ea0 0x7f55500ceef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5540009870 tx=0x7f554002b8f0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7fa5edffb640 1 -- 172.21.15.72:0/2853879556 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e802cdd0 con 0x7fa5f00d0d70 2023-12-12T09:52:42.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7fa5edffb640 1 -- 172.21.15.72:0/2853879556 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8034640 con 0x7fa5f00d0d70 2023-12-12T09:52:42.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7f554ffff640 1 -- 172.21.15.72:0/1269045951 shutdown_connections 2023-12-12T09:52:42.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7f554ffff640 1 --2- 172.21.15.72:0/1269045951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5550071ea0 0x7f55500ceef0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7f554ffff640 1 --2- 172.21.15.72:0/1269045951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f555015bc00 0x7f555006c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7f554ffff640 1 -- 172.21.15.72:0/1269045951 >> 172.21.15.72:0/1269045951 conn(0x7f555006a460 msgr2=0x7f555006a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7efedeffd640 1 --2- 172.21.15.72:0/1127280139 >> v2:172.21.15.113:3300/0 conn(0x7efed8049bd0 0x7efed80eb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7efedeffd640 1 -- 172.21.15.72:0/1127280139 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8054b60 con 0x7efed80362c0 2023-12-12T09:52:42.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7efedf7fe640 1 --2- 172.21.15.72:0/1127280139 >> v2:172.21.15.113:3300/0 conn(0x7efed8049bd0 0x7efed80eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.279+0000 7f554ffff640 1 -- 172.21.15.72:0/1269045951 shutdown_connections 2023-12-12T09:52:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.280+0000 7fa5effff640 1 -- 172.21.15.72:0/2853879556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 msgr2=0x7fa5f00b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.280+0000 7fa5effff640 1 --2- 172.21.15.72:0/2853879556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 0x7fa5f00b0e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009d90 tx=0x7fa5e802bec0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.280+0000 7f554ffff640 1 -- 172.21.15.72:0/1269045951 wait complete. 2023-12-12T09:52:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.281+0000 7efedeffd640 1 --2- 172.21.15.72:0/1127280139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed80362c0 0x7efed800dc30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efed800ed10 tx=0x7efed402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6272c68930f0754 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.281+0000 7efe2df68640 1 Processor -- start 2023-12-12T09:52:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7efeddffb640 1 -- 172.21.15.72:0/1127280139 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4031870 con 0x7efed80362c0 2023-12-12T09:52:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.281+0000 7efe2df68640 1 -- start start 2023-12-12T09:52:42.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7fa5effff640 1 -- 172.21.15.72:0/2853879556 shutdown_connections 2023-12-12T09:52:42.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7fa5effff640 1 --2- 172.21.15.72:0/2853879556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 0x7fa5f00b0e70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7fa5effff640 1 -- 172.21.15.72:0/2853879556 >> 172.21.15.72:0/2853879556 conn(0x7fa5f009c260 msgr2=0x7fa5f00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7efeddffb640 1 -- 172.21.15.72:0/1127280139 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed40319d0 con 0x7efed80362c0 2023-12-12T09:52:42.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7efeddffb640 1 -- 172.21.15.72:0/1127280139 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4031d60 con 0x7efed80362c0 2023-12-12T09:52:42.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7fa5effff640 1 -- 172.21.15.72:0/2853879556 shutdown_connections 2023-12-12T09:52:42.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7fa5effff640 1 -- 172.21.15.72:0/2853879556 wait complete. 2023-12-12T09:52:42.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.282+0000 7efe2df68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe280ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efe2df68640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efe280708d0 0x7efe2816b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efe2df68640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe280cde40 con 0x7efe281626b0 2023-12-12T09:52:42.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efe2df68640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe280cd040 con 0x7efe280708d0 2023-12-12T09:52:42.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efe2df68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2815f640 con 0x7efe2815f260 2023-12-12T09:52:42.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efedffff640 1 -- 172.21.15.72:0/1127280139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed80362c0 msgr2=0x7efed800dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efedffff640 1 --2- 172.21.15.72:0/1127280139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed80362c0 0x7efed800dc30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efed800ed10 tx=0x7efed402b910 comp rx=0 tx=0).stop 2023-12-12T09:52:42.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7f554ffff640 1 Processor -- start 2023-12-12T09:52:42.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efe26ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7efe281626b0 0x7efe280701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52476/0 (socket says 172.21.15.72:52476) 2023-12-12T09:52:42.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efe277fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe280ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efe26ffd640 1 -- 172.21.15.72:0/2398110012 learned_addr learned my addr 172.21.15.72:0/2398110012 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:42.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.283+0000 7efe27fff640 1 --2- 172.21.15.72:0/2398110012 >> v2:172.21.15.72:3300/0 conn(0x7efe280708d0 0x7efe2816b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efedffff640 1 -- 172.21.15.72:0/1127280139 shutdown_connections 2023-12-12T09:52:42.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efedffff640 1 --2- 172.21.15.72:0/1127280139 >> v2:172.21.15.113:3300/0 conn(0x7efed8049bd0 0x7efed80eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efedffff640 1 --2- 172.21.15.72:0/1127280139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed8056750 0x7efed800e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efedffff640 1 --2- 172.21.15.72:0/1127280139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed80362c0 0x7efed800dc30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efedffff640 1 -- 172.21.15.72:0/1127280139 >> 172.21.15.72:0/1127280139 conn(0x7efed8056cb0 msgr2=0x7efed80570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7f554ffff640 1 -- start start 2023-12-12T09:52:42.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efedffff640 1 -- 172.21.15.72:0/1127280139 shutdown_connections 2023-12-12T09:52:42.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efe27fff640 1 -- 172.21.15.72:0/2398110012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 msgr2=0x7efe280ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efe27fff640 1 --2- 172.21.15.72:0/2398110012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe280ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.285+0000 7fa5effff640 1 Processor -- start 2023-12-12T09:52:42.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.285+0000 7fa5effff640 1 -- start start 2023-12-12T09:52:42.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.286+0000 7f554ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5550071ea0 0x7f55501649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efe267fc640 1 -- 172.21.15.72:0/2398110012 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667769416 0 0) 0x7efe280cde40 con 0x7efe281626b0 2023-12-12T09:52:42.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efe27fff640 1 -- 172.21.15.72:0/2398110012 >> v1:172.21.15.72:6789/0 conn(0x7efe281626b0 legacy=0x7efe280701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.286+0000 7f554ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f555015b820 0x7f55501631e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efedffff640 1 -- 172.21.15.72:0/1127280139 wait complete. 2023-12-12T09:52:42.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.286+0000 7f554ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f555015bc00 0x7f5550154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.284+0000 7efe27fff640 1 -- 172.21.15.72:0/2398110012 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28162b10 con 0x7efe280708d0 2023-12-12T09:52:42.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7f554ffff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5550155650 con 0x7f555015bc00 2023-12-12T09:52:42.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7f554e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f555015b820 0x7f55501631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7efedffff640 1 Processor -- start 2023-12-12T09:52:42.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7fa5effff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00ce5b0 0x7fa5f00cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7f554e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f555015b820 0x7f55501631e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57962/0 (socket says 172.21.15.72:57962) 2023-12-12T09:52:42.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7f554e7fc640 1 -- 172.21.15.72:0/2522105095 learned_addr learned my addr 172.21.15.72:0/2522105095 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7efedffff640 1 -- start start 2023-12-12T09:52:42.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7fa5effff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 0x7fa5f0143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7fa5effff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00d13c0 0x7fa5f01f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7fa5effff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f0143890 con 0x7fa5f00d0d70 2023-12-12T09:52:42.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7fa5effff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f0156fa0 con 0x7fa5f00ce5b0 2023-12-12T09:52:42.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7fa5effff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00ccf20 con 0x7fa5f00d13c0 2023-12-12T09:52:42.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00ce5b0 0x7fa5f00cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7fa5ee7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 0x7fa5f0143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00ce5b0 0x7fa5f00cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57970/0 (socket says 172.21.15.72:57970) 2023-12-12T09:52:42.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7fa5eeffd640 1 -- 172.21.15.72:0/4221264646 learned_addr learned my addr 172.21.15.72:0/4221264646 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7fa5ef7fe640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00d13c0 0x7fa5f01f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7f554effd640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5550071ea0 0x7f55501649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.287+0000 7f554f7fe640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f555015bc00 0x7f5550154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7f554ffff640 1 -- 172.21.15.72:0/2522105095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5550163720 con 0x7f555015b820 2023-12-12T09:52:42.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7f554ffff640 1 -- 172.21.15.72:0/2522105095 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5550163950 con 0x7f5550071ea0 2023-12-12T09:52:42.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7efe27fff640 1 --2- 172.21.15.72:0/2398110012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe280708d0 0x7efe2816b1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe20009dc0 tx=0x7efe2002be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d695402766fc2e84 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.288+0000 7efe267fc640 1 -- 172.21.15.72:0/2398110012 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2002cc30 con 0x7efe280708d0 2023-12-12T09:52:42.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7efe267fc640 1 -- 172.21.15.72:0/2398110012 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2002cd90 con 0x7efe280708d0 2023-12-12T09:52:42.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7efe267fc640 1 -- 172.21.15.72:0/2398110012 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20034660 con 0x7efe280708d0 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7fa5ee7fc640 1 -- 172.21.15.72:0/4221264646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00d13c0 msgr2=0x7fa5f01f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7fa5ee7fc640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00d13c0 0x7fa5f01f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7fa5ee7fc640 1 -- 172.21.15.72:0/4221264646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00ce5b0 msgr2=0x7fa5f00cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7fa5ee7fc640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00ce5b0 0x7fa5f00cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7fa5ee7fc640 1 -- 172.21.15.72:0/4221264646 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e8009a70 con 0x7fa5f00d0d70 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7fa5ee7fc640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 0x7fa5f0143350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400c340 tx=0x7fa5e400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.289+0000 7fa5d77fe640 1 -- 172.21.15.72:0/4221264646 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40052d0 con 0x7fa5f00d0d70 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.290+0000 7fa5d77fe640 1 -- 172.21.15.72:0/4221264646 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4005450 con 0x7fa5f00d0d70 2023-12-12T09:52:42.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.290+0000 7fa5d77fe640 1 -- 172.21.15.72:0/4221264646 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4003830 con 0x7fa5f00d0d70 2023-12-12T09:52:42.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.290+0000 7efe2df68640 1 -- 172.21.15.72:0/2398110012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe280708d0 msgr2=0x7efe2816b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.290+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f01585e0 con 0x7fa5f00d0d70 2023-12-12T09:52:42.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.290+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f0158ba0 con 0x7fa5f00d0d70 2023-12-12T09:52:42.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.290+0000 7efe2df68640 1 --2- 172.21.15.72:0/2398110012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe280708d0 0x7efe2816b1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe20009dc0 tx=0x7efe2002be80 comp rx=0 tx=0).stop 2023-12-12T09:52:42.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7f554e7fc640 1 -- 172.21.15.72:0/2522105095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5550071ea0 msgr2=0x7f55501649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7efe2df68640 1 -- 172.21.15.72:0/2398110012 shutdown_connections 2023-12-12T09:52:42.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7efe2df68640 1 --2- 172.21.15.72:0/2398110012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe280708d0 0x7efe2816b1c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7efe2df68640 1 --2- 172.21.15.72:0/2398110012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe280ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7efe2df68640 1 -- 172.21.15.72:0/2398110012 >> 172.21.15.72:0/2398110012 conn(0x7efe280b0a90 msgr2=0x7efe280b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.292+0000 7efedffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed80362c0 0x7efed8185540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.292+0000 7efedffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed8049bd0 0x7efed804c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.292+0000 7efedffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efed8056750 0x7efed804c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efedffff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8037070 con 0x7efed8049bd0 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efedffff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed800abd0 con 0x7efed8056750 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efedffff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed8036740 con 0x7efed80362c0 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efedeffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed80362c0 0x7efed8185540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efedeffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed80362c0 0x7efed8185540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49670/0 (socket says 172.21.15.72:49670) 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efedeffd640 1 -- 172.21.15.72:0/2264913065 learned_addr learned my addr 172.21.15.72:0/2264913065 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7f554e7fc640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5550071ea0 0x7f55501649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7f554e7fc640 1 -- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f555015bc00 msgr2=0x7f5550154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7f554e7fc640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f555015bc00 0x7f5550154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7f554e7fc640 1 -- 172.21.15.72:0/2522105095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400094d0 con 0x7f555015b820 2023-12-12T09:52:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.291+0000 7f554f7fe640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f555015bc00 0x7f5550154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.292+0000 7f554e7fc640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f555015b820 0x7f55501631e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f554400e870 tx=0x7f554400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7f55377fe640 1 -- 172.21.15.72:0/2522105095 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55440094f0 con 0x7f555015b820 2023-12-12T09:52:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efe2df68640 1 -- 172.21.15.72:0/2398110012 shutdown_connections 2023-12-12T09:52:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7f55377fe640 1 -- 172.21.15.72:0/2522105095 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554400c940 con 0x7f555015b820 2023-12-12T09:52:42.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7f55377fe640 1 -- 172.21.15.72:0/2522105095 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400ccf0 con 0x7f555015b820 2023-12-12T09:52:42.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efe2df68640 1 -- 172.21.15.72:0/2398110012 wait complete. 2023-12-12T09:52:42.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.293+0000 7efede7fc640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed8049bd0 0x7efed804c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.294+0000 7efedf7fe640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efed8056750 0x7efed804c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.294+0000 7efede7fc640 1 -- 172.21.15.72:0/2264913065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed80362c0 msgr2=0x7efed8185540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.294+0000 7efede7fc640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed80362c0 0x7efed8185540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.294+0000 7efede7fc640 1 -- 172.21.15.72:0/2264913065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efed8056750 msgr2=0x7efed804c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.294+0000 7efede7fc640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efed8056750 0x7efed804c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.295+0000 7efede7fc640 1 -- 172.21.15.72:0/2264913065 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40094d0 con 0x7efed8049bd0 2023-12-12T09:52:42.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.295+0000 7fa5d77fe640 1 -- 172.21.15.72:0/4221264646 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa5e4003990 con 0x7fa5f00d0d70 2023-12-12T09:52:42.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.294+0000 7f554ffff640 1 -- 172.21.15.72:0/2522105095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5550163c30 con 0x7f555015b820 2023-12-12T09:52:42.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.294+0000 7f554ffff640 1 -- 172.21.15.72:0/2522105095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55501641f0 con 0x7f555015b820 2023-12-12T09:52:42.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.295+0000 7efede7fc640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed8049bd0 0x7efed804c160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efed000e850 tx=0x7efed000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.295+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d0005250 con 0x7fa5f00d0d70 2023-12-12T09:52:42.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.296+0000 7efec77fe640 1 -- 172.21.15.72:0/2264913065 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00095b0 con 0x7efed8049bd0 2023-12-12T09:52:42.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.296+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed8036a20 con 0x7efed8049bd0 2023-12-12T09:52:42.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.297+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed80ea7a0 con 0x7efed8049bd0 2023-12-12T09:52:42.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.298+0000 7efe2df68640 1 Processor -- start 2023-12-12T09:52:42.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.299+0000 7efe2df68640 1 -- start start 2023-12-12T09:52:42.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.299+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed80073a0 con 0x7efed8049bd0 2023-12-12T09:52:42.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.300+0000 7f55357fa640 1 -- 172.21.15.72:0/2522105095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555006cc60 con 0x7f555015b820 2023-12-12T09:52:42.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.300+0000 7f55377fe640 1 -- 172.21.15.72:0/2522105095 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5544013470 con 0x7f555015b820 2023-12-12T09:52:42.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.300+0000 7fa5d77fe640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa5dc06ae40 0x7fa5dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.301+0000 7fa5eeffd640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa5dc06ae40 0x7fa5dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.301+0000 7efe2df68640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe280708d0 0x7efe2815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.301+0000 7efe2df68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe2814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.301+0000 7efe2df68640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe281626b0 0x7efe281e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.301+0000 7efe2df68640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe280b3700 con 0x7efe281626b0 2023-12-12T09:52:42.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.301+0000 7efe2df68640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe281ea300 con 0x7efe280708d0 2023-12-12T09:52:42.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.301+0000 7efe2df68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe280b2af0 con 0x7efe2815f260 2023-12-12T09:52:42.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.301+0000 7f55377fe640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f553c06af10 0x7f553c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.302+0000 7efe26ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe2814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.302+0000 7f55377fe640 1 -- 172.21.15.72:0/2522105095 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5544089950 con 0x7f555015b820 2023-12-12T09:52:42.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.302+0000 7efe26ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe2814a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49676/0 (socket says 172.21.15.72:49676) 2023-12-12T09:52:42.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.302+0000 7efe26ffd640 1 -- 172.21.15.72:0/4171090071 learned_addr learned my addr 172.21.15.72:0/4171090071 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.302+0000 7efe27fff640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe281626b0 0x7efe281e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.302+0000 7efe277fe640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe280708d0 0x7efe2815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.303+0000 7f554effd640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f553c06af10 0x7f553c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.303+0000 7fa5d77fe640 1 -- 172.21.15.72:0/4221264646 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa5e4088e10 con 0x7fa5f00d0d70 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.303+0000 7efe27fff640 1 -- 172.21.15.72:0/4171090071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 msgr2=0x7efe2814a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.303+0000 7efe27fff640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe2814a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.303+0000 7efe27fff640 1 -- 172.21.15.72:0/4171090071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe280708d0 msgr2=0x7efe2815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.303+0000 7efe27fff640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe280708d0 0x7efe2815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.303+0000 7efe27fff640 1 -- 172.21.15.72:0/4171090071 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20009a70 con 0x7efe281626b0 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.304+0000 7efe26ffd640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe2814a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.304+0000 7efe27fff640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe281626b0 0x7efe281e9dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe20033000 tx=0x7efe20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.305+0000 7f554effd640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f553c06af10 0x7f553c06d3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55480012e0 tx=0x7f55480016f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.305+0000 7fa5eeffd640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa5dc06ae40 0x7fa5dc06d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00011e0 tx=0x7fa5e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.307+0000 7efec77fe640 1 -- 172.21.15.72:0/2264913065 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0005990 con 0x7efed8049bd0 2023-12-12T09:52:42.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.308+0000 7efec77fe640 1 -- 172.21.15.72:0/2264913065 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0005d00 con 0x7efed8049bd0 2023-12-12T09:52:42.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.308+0000 7efe1ffff640 1 -- 172.21.15.72:0/4171090071 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20040650 con 0x7efe281626b0 2023-12-12T09:52:42.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.308+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe280b2dd0 con 0x7efe281626b0 2023-12-12T09:52:42.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.308+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe280b3390 con 0x7efe281626b0 2023-12-12T09:52:42.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.308+0000 7fa5d77fe640 1 -- 172.21.15.72:0/4221264646 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa5e4053ce0 con 0x7fa5f00d0d70 2023-12-12T09:52:42.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.310+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe280b3870 con 0x7efe281626b0 2023-12-12T09:52:42.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.311+0000 7efe1ffff640 1 -- 172.21.15.72:0/4171090071 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe200407b0 con 0x7efe281626b0 2023-12-12T09:52:42.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.311+0000 7efe1ffff640 1 -- 172.21.15.72:0/4171090071 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20040b20 con 0x7efe281626b0 2023-12-12T09:52:42.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.311+0000 7efec77fe640 1 -- 172.21.15.72:0/2264913065 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efed000c940 con 0x7efed8049bd0 2023-12-12T09:52:42.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.313+0000 7efec77fe640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efecc06ae40 0x7efecc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.313+0000 7efec77fe640 1 -- 172.21.15.72:0/2264913065 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efed008a1b0 con 0x7efed8049bd0 2023-12-12T09:52:42.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.314+0000 7efedeffd640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efecc06ae40 0x7efecc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.315+0000 7efedeffd640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efecc06ae40 0x7efecc06d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.316+0000 7f55377fe640 1 -- 172.21.15.72:0/2522105095 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55440547a0 con 0x7f555015b820 2023-12-12T09:52:42.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.318+0000 7efec77fe640 1 -- 172.21.15.72:0/2264913065 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efed00550b0 con 0x7efed8049bd0 2023-12-12T09:52:42.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.321+0000 7efe1ffff640 1 -- 172.21.15.72:0/4171090071 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe20040c80 con 0x7efe281626b0 2023-12-12T09:52:42.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.322+0000 7efe1ffff640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe0806ae40 0x7efe0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.322+0000 7efe1ffff640 1 -- 172.21.15.72:0/4171090071 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efe200acc40 con 0x7efe281626b0 2023-12-12T09:52:42.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.323+0000 7efe277fe640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe0806ae40 0x7efe0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.328+0000 7efe277fe640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe0806ae40 0x7efe0806d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efe10006fd0 tx=0x7efe10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.329+0000 7efe1ffff640 1 -- 172.21.15.72:0/4171090071 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe20077b00 con 0x7efe281626b0 2023-12-12T09:52:42.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.567+0000 7f58c0ff9640 1 -- 172.21.15.72:0/1611596252 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f58ac0057d0 con 0x7f58cc1051b0 2023-12-12T09:52:42.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.567+0000 7f58c2ffd640 1 -- 172.21.15.72:0/1611596252 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f58c807ee60 con 0x7f58cc1051b0 2023-12-12T09:52:42.572 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-12-12T09:52:42.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.571+0000 7f58c0ff9640 1 -- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f58b406af00 msgr2=0x7f58b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.571+0000 7f58c0ff9640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f58b406af00 0x7f58b406d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f58c4006fd0 tx=0x7f58c4008040 comp rx=0 tx=0).stop 2023-12-12T09:52:42.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.571+0000 7f58c0ff9640 1 -- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc1051b0 msgr2=0x7f58cc006d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.571+0000 7f58c0ff9640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc1051b0 0x7f58cc006d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0509e0 tx=0x7f58c80046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.572+0000 7f58c0ff9640 1 -- 172.21.15.72:0/1611596252 shutdown_connections 2023-12-12T09:52:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.572+0000 7f58c0ff9640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f58b406af00 0x7f58b406d3c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.572+0000 7f58c0ff9640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc1051b0 0x7f58cc006d70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.572+0000 7f58c0ff9640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58cc053fe0 0x7f58cc105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.572+0000 7f58c0ff9640 1 --2- 172.21.15.72:0/1611596252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58cc037bf0 0x7f58cc04a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.572+0000 7f58c0ff9640 1 -- 172.21.15.72:0/1611596252 >> 172.21.15.72:0/1611596252 conn(0x7f58cc009f80 msgr2=0x7f58cc04bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.573+0000 7f58c0ff9640 1 -- 172.21.15.72:0/1611596252 shutdown_connections 2023-12-12T09:52:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.573+0000 7f58c0ff9640 1 -- 172.21.15.72:0/1611596252 wait complete. 2023-12-12T09:52:42.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.2 2023-12-12T09:52:42.601 DEBUG:teuthology.parallel:result is None 2023-12-12T09:52:42.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.606+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fda6815e0a0 con 0x7fda680700e0 2023-12-12T09:52:42.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.607+0000 7fda64ff9640 1 -- 172.21.15.72:0/2340559677 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda6815e0a0 con 0x7fda680700e0 2023-12-12T09:52:42.609 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-12-12T09:52:42.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.610+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fda4c06af10 msgr2=0x7fda4c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.610+0000 7fda6e01c640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fda4c06af10 0x7fda4c06d3d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda6815f360 tx=0x7fda50009040 comp rx=0 tx=0).stop 2023-12-12T09:52:42.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.610+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680700e0 msgr2=0x7fda681603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.610+0000 7fda6e01c640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680700e0 0x7fda681603d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fda5800d5f0 tx=0x7fda5800dab0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.611+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 shutdown_connections 2023-12-12T09:52:42.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.611+0000 7fda6e01c640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fda4c06af10 0x7fda4c06d3d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.611+0000 7fda6e01c640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda6815d130 0x7fda68155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.611+0000 7fda6e01c640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda680b3cc0 0x7fda68155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.611+0000 7fda6e01c640 1 --2- 172.21.15.72:0/2340559677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda680700e0 0x7fda681603d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.611+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 >> 172.21.15.72:0/2340559677 conn(0x7fda680b0940 msgr2=0x7fda680af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.611+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 shutdown_connections 2023-12-12T09:52:42.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.611+0000 7fda6e01c640 1 -- 172.21.15.72:0/2340559677 wait complete. 2023-12-12T09:52:42.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.0 2023-12-12T09:52:42.639 DEBUG:teuthology.parallel:result is None 2023-12-12T09:52:42.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.638+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f755800c4b0 con 0x7f7558006170 2023-12-12T09:52:42.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.639+0000 7f7556ffd640 1 -- 172.21.15.72:0/2945894801 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f755800c4b0 con 0x7f7558006170 2023-12-12T09:52:42.640 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-12-12T09:52:42.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.640+0000 7f5f6dffb640 1 -- 172.21.15.72:0/304988225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5f600057d0 con 0x7f5f880b3e70 2023-12-12T09:52:42.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.641+0000 7f5f6ffff640 1 -- 172.21.15.72:0/304988225 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f7c04a020 con 0x7f5f880b3e70 2023-12-12T09:52:42.643 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-12-12T09:52:42.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f754006af10 msgr2=0x7f754006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f754006af10 0x7f754006d3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f755804eaf0 tx=0x7f7548007ef0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7558006170 msgr2=0x7f75580e1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7558006170 0x7f75580e1100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75600697f0 tx=0x7f7560069ea0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 shutdown_connections 2023-12-12T09:52:42.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f754006af10 0x7f754006d3d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75580eb410 0x7f7558189a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75580eb030 0x7f7558189520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 --2- 172.21.15.72:0/2945894801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7558006170 0x7f75580e1100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.642+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 >> 172.21.15.72:0/2945894801 conn(0x7f75580559b0 msgr2=0x7f755803c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.643+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 shutdown_connections 2023-12-12T09:52:42.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.643+0000 7f7564e52640 1 -- 172.21.15.72:0/2945894801 wait complete. 2023-12-12T09:52:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 -- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5f7406ae70 msgr2=0x7f5f7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5f7406ae70 0x7f5f7406d330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f880b5dd0 tx=0x7f5f78003990 comp rx=0 tx=0).stop 2023-12-12T09:52:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 -- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f880b3e70 msgr2=0x7f5f880cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f880b3e70 0x7f5f880cf2a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009890 tx=0x7f5f7c0046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 -- 172.21.15.72:0/304988225 shutdown_connections 2023-12-12T09:52:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5f7406ae70 0x7f5f7406d330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f881606d0 0x7f5f881f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f880cfcd0 0x7f5f880c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 --2- 172.21.15.72:0/304988225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f880b3e70 0x7f5f880cf2a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 -- 172.21.15.72:0/304988225 >> 172.21.15.72:0/304988225 conn(0x7f5f880d2f60 msgr2=0x7f5f880b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.644+0000 7f5f6dffb640 1 -- 172.21.15.72:0/304988225 shutdown_connections 2023-12-12T09:52:42.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.645+0000 7f5f6dffb640 1 -- 172.21.15.72:0/304988225 wait complete. 2023-12-12T09:52:42.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.653+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa5d0005500 con 0x7fa5f00d0d70 2023-12-12T09:52:42.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.653+0000 7fa5d77fe640 1 -- 172.21.15.72:0/4221264646 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5e4017090 con 0x7fa5f00d0d70 2023-12-12T09:52:42.655 INFO:teuthology.orchestra.run.smithi072.stdout:55834574860 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.655+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa5dc06ae40 msgr2=0x7fa5dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.655+0000 7fa5effff640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa5dc06ae40 0x7fa5dc06d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00011e0 tx=0x7fa5e0009040 comp rx=0 tx=0).stop 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.655+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 msgr2=0x7fa5f0143350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.655+0000 7fa5effff640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 0x7fa5f0143350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400c340 tx=0x7fa5e400c800 comp rx=0 tx=0).stop 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.656+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 shutdown_connections 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.656+0000 7fa5effff640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa5dc06ae40 0x7fa5dc06d300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.656+0000 7fa5effff640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5f00d13c0 0x7fa5f01f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.656+0000 7fa5effff640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5f00d0d70 0x7fa5f0143350 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.656+0000 7fa5effff640 1 --2- 172.21.15.72:0/4221264646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5f00ce5b0 0x7fa5f00cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.656+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 >> 172.21.15.72:0/4221264646 conn(0x7fa5f009c260 msgr2=0x7fa5f00b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.656+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 shutdown_connections 2023-12-12T09:52:42.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.656+0000 7fa5effff640 1 -- 172.21.15.72:0/4221264646 wait complete. 2023-12-12T09:52:42.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.1 2023-12-12T09:52:42.673 DEBUG:teuthology.parallel:result is None 2023-12-12T09:52:42.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.3 2023-12-12T09:52:42.674 DEBUG:teuthology.parallel:result is None 2023-12-12T09:52:42.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.7 2023-12-12T09:52:42.676 DEBUG:teuthology.parallel:result is None 2023-12-12T09:52:42.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.685+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7efe280745d0 con 0x7efe281626b0 2023-12-12T09:52:42.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.685+0000 7efe1ffff640 1 -- 172.21.15.72:0/4171090071 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe2007b120 con 0x7efe281626b0 2023-12-12T09:52:42.687 INFO:teuthology.orchestra.run.smithi072.stdout:55834574859 2023-12-12T09:52:42.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.687+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe0806ae40 msgr2=0x7efe0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.687+0000 7efe2df68640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe0806ae40 0x7efe0806d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efe10006fd0 tx=0x7efe10008040 comp rx=0 tx=0).stop 2023-12-12T09:52:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.687+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe281626b0 msgr2=0x7efe281e9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.687+0000 7efe2df68640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe281626b0 0x7efe281e9dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe20033000 tx=0x7efe20004a40 comp rx=0 tx=0).stop 2023-12-12T09:52:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.688+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 shutdown_connections 2023-12-12T09:52:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.688+0000 7efe2df68640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe0806ae40 0x7efe0806d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.688+0000 7efe2df68640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe281626b0 0x7efe281e9dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.688+0000 7efe2df68640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe2815f260 0x7efe2814a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.688+0000 7efe2df68640 1 --2- 172.21.15.72:0/4171090071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe280708d0 0x7efe2815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.688+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 >> 172.21.15.72:0/4171090071 conn(0x7efe280b0a90 msgr2=0x7efe280caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.688+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 shutdown_connections 2023-12-12T09:52:42.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.688+0000 7efe2df68640 1 -- 172.21.15.72:0/4171090071 wait complete. 2023-12-12T09:52:42.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.693+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7efed8039d50 con 0x7efed8049bd0 2023-12-12T09:52:42.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.693+0000 7efec77fe640 1 -- 172.21.15.72:0/2264913065 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efed0017020 con 0x7efed8049bd0 2023-12-12T09:52:42.695 INFO:teuthology.orchestra.run.smithi072.stdout:55834574859 2023-12-12T09:52:42.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efecc06ae40 msgr2=0x7efecc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efecc06ae40 0x7efecc06d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed403c040 comp rx=0 tx=0).stop 2023-12-12T09:52:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed8049bd0 msgr2=0x7efed804c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed8049bd0 0x7efed804c160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efed000e850 tx=0x7efed000ed10 comp rx=0 tx=0).stop 2023-12-12T09:52:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 shutdown_connections 2023-12-12T09:52:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efecc06ae40 0x7efecc06d300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efed8056750 0x7efed804c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efed8049bd0 0x7efed804c160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 --2- 172.21.15.72:0/2264913065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efed80362c0 0x7efed8185540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.695+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 >> 172.21.15.72:0/2264913065 conn(0x7efed8056cb0 msgr2=0x7efed800dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.696+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 shutdown_connections 2023-12-12T09:52:42.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.696+0000 7efedffff640 1 -- 172.21.15.72:0/2264913065 wait complete. 2023-12-12T09:52:42.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.700+0000 7f55357fa640 1 -- 172.21.15.72:0/2522105095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5550074450 con 0x7f555015b820 2023-12-12T09:52:42.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.700+0000 7f55377fe640 1 -- 172.21.15.72:0/2522105095 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5544057dc0 con 0x7f555015b820 2023-12-12T09:52:42.702 INFO:teuthology.orchestra.run.smithi072.stdout:55834574860 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 -- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f553c06af10 msgr2=0x7f553c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f553c06af10 0x7f553c06d3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55480012e0 tx=0x7f55480016f0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 -- 172.21.15.72:0/2522105095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f555015b820 msgr2=0x7f55501631e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f555015b820 0x7f55501631e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f554400e870 tx=0x7f554400ed30 comp rx=0 tx=0).stop 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 -- 172.21.15.72:0/2522105095 shutdown_connections 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f553c06af10 0x7f553c06d3d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f555015bc00 0x7f5550154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f555015b820 0x7f55501631e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 --2- 172.21.15.72:0/2522105095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5550071ea0 0x7f55501649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 -- 172.21.15.72:0/2522105095 >> 172.21.15.72:0/2522105095 conn(0x7f555006a460 msgr2=0x7f55500b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 -- 172.21.15.72:0/2522105095 shutdown_connections 2023-12-12T09:52:42.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.703+0000 7f55357fa640 1 -- 172.21.15.72:0/2522105095 wait complete. 2023-12-12T09:52:42.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2023-12-12T09:52:42.707 DEBUG:teuthology.parallel:result is None 2023-12-12T09:52:42.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2023-12-12T09:52:42.714 DEBUG:teuthology.parallel:result is None 2023-12-12T09:52:42.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.4 2023-12-12T09:52:42.721 DEBUG:teuthology.parallel:result is None 2023-12-12T09:52:42.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-12T09:52:42.721 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:52:42.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.964+0000 7f9bc3b01640 1 Processor -- start 2023-12-12T09:52:42.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.964+0000 7f9bc3b01640 1 -- start start 2023-12-12T09:52:42.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.965+0000 7f9bc3b01640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc1467c0 0x7f9bbc0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.965+0000 7f9bc3b01640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9bbc146ba0 0x7f9bbc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.965+0000 7f9bc3b01640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc06d510 con 0x7f9bbc06c7e0 2023-12-12T09:52:42.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.965+0000 7f9bc3b01640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc0ac860 con 0x7f9bbc146ba0 2023-12-12T09:52:42.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.965+0000 7f9bc3b01640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc06d7f0 con 0x7f9bbc1467c0 2023-12-12T09:52:42.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc109c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9bbc146ba0 0x7f9bbc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc209e640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f9bbc06c7e0 0x7f9bbc06cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52480/0 (socket says 172.21.15.72:52480) 2023-12-12T09:52:42.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc209e640 1 -- 172.21.15.72:0/1983908165 learned_addr learned my addr 172.21.15.72:0/1983908165 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:42.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc189d640 1 --2- 172.21.15.72:0/1983908165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc1467c0 0x7f9bbc0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc109c640 1 -- 172.21.15.72:0/1983908165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc1467c0 msgr2=0x7f9bbc0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:42.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc109c640 1 --2- 172.21.15.72:0/1983908165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc1467c0 0x7f9bbc0cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc109c640 1 -- 172.21.15.72:0/1983908165 >> v1:172.21.15.72:6789/0 conn(0x7f9bbc06c7e0 legacy=0x7f9bbc06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc109c640 1 -- 172.21.15.72:0/1983908165 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc06dad0 con 0x7f9bbc146ba0 2023-12-12T09:52:42.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.966+0000 7f9bc089b640 1 -- 172.21.15.72:0/1983908165 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920143758 0 0) 0x7f9bbc06d510 con 0x7f9bbc06c7e0 2023-12-12T09:52:42.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.967+0000 7f9bc109c640 1 --2- 172.21.15.72:0/1983908165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc146ba0 0x7f9bbc0cce30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0097a0 tx=0x7f9bac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40efa866687428b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc089b640 1 -- 172.21.15.72:0/1983908165 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac031830 con 0x7f9bbc146ba0 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc089b640 1 -- 172.21.15.72:0/1983908165 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac031990 con 0x7f9bbc146ba0 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc089b640 1 -- 172.21.15.72:0/1983908165 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac031d20 con 0x7f9bbc146ba0 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc3b01640 1 -- 172.21.15.72:0/1983908165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc146ba0 msgr2=0x7f9bbc0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/1983908165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc146ba0 0x7f9bbc0cce30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0097a0 tx=0x7f9bac02b8d0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc3b01640 1 -- 172.21.15.72:0/1983908165 shutdown_connections 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/1983908165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc146ba0 0x7f9bbc0cce30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/1983908165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc1467c0 0x7f9bbc0cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc3b01640 1 -- 172.21.15.72:0/1983908165 >> 172.21.15.72:0/1983908165 conn(0x7f9bbc0b0a90 msgr2=0x7f9bbc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc3b01640 1 -- 172.21.15.72:0/1983908165 shutdown_connections 2023-12-12T09:52:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.968+0000 7f9bc3b01640 1 -- 172.21.15.72:0/1983908165 wait complete. 2023-12-12T09:52:42.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.970+0000 7f9bc3b01640 1 Processor -- start 2023-12-12T09:52:42.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.970+0000 7f9bc3b01640 1 -- start start 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc3b01640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bbc06c7e0 0x7f9bbc1f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc3b01640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc1467c0 0x7f9bbc0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc3b01640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc146ba0 0x7f9bbc1324c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc3b01640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1f37d0 con 0x7f9bbc1467c0 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc3b01640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1e7550 con 0x7f9bbc06c7e0 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc3b01640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1e83b0 con 0x7f9bbc146ba0 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc109c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc1467c0 0x7f9bbc0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc109c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc1467c0 0x7f9bbc0b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40648/0 (socket says 172.21.15.72:40648) 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.971+0000 7f9bc109c640 1 -- 172.21.15.72:0/4075705155 learned_addr learned my addr 172.21.15.72:0/4075705155 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:42.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc189d640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bbc06c7e0 0x7f9bbc1f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc109c640 1 -- 172.21.15.72:0/4075705155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc146ba0 msgr2=0x7f9bbc1324c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc109c640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc146ba0 0x7f9bbc1324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc109c640 1 -- 172.21.15.72:0/4075705155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bbc06c7e0 msgr2=0x7f9bbc1f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc109c640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bbc06c7e0 0x7f9bbc1f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc109c640 1 -- 172.21.15.72:0/4075705155 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac009400 con 0x7f9bbc1467c0 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc109c640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc1467c0 0x7f9bbc0b47d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0097a0 tx=0x7f9bac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bb27fc640 1 -- 172.21.15.72:0/4075705155 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac004230 con 0x7f9bbc1467c0 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bb27fc640 1 -- 172.21.15.72:0/4075705155 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac004390 con 0x7f9bbc1467c0 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bb27fc640 1 -- 172.21.15.72:0/4075705155 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac03a870 con 0x7f9bbc1467c0 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc1e77a0 con 0x7f9bbc1467c0 2023-12-12T09:52:42.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.972+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc1e7d60 con 0x7f9bbc1467c0 2023-12-12T09:52:42.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.974+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc0ce7c0 con 0x7f9bbc1467c0 2023-12-12T09:52:42.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.974+0000 7f9bb27fc640 1 -- 172.21.15.72:0/4075705155 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9bac03a9d0 con 0x7f9bbc1467c0 2023-12-12T09:52:42.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.975+0000 7f9bb27fc640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ba006ae70 0x7f9ba006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.975+0000 7f9bb27fc640 1 -- 172.21.15.72:0/4075705155 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9bac0abc30 con 0x7f9bbc1467c0 2023-12-12T09:52:42.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.975+0000 7f9bc189d640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ba006ae70 0x7f9ba006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.976+0000 7f9bc189d640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ba006ae70 0x7f9ba006d330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8006fd0 tx=0x7f9bb8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:42.979+0000 7f9bb27fc640 1 -- 172.21.15.72:0/4075705155 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bac0769e0 con 0x7f9bbc1467c0 2023-12-12T09:52:43.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.152+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bbc0745d0 con 0x7f9ba006ae70 2023-12-12T09:52:43.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.178+0000 7f9bb27fc640 1 -- 172.21.15.72:0/4075705155 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345125 (secure 0 0 0) 0x7f9bbc0745d0 con 0x7f9ba006ae70 2023-12-12T09:52:43.185 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:43.187 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.185+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ba006ae70 msgr2=0x7f9ba006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.185+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ba006ae70 0x7f9ba006d330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8006fd0 tx=0x7f9bb8008040 comp rx=0 tx=0).stop 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.185+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc1467c0 msgr2=0x7f9bbc0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.185+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc1467c0 0x7f9bbc0b47d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0097a0 tx=0x7f9bac0046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.186+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 shutdown_connections 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.186+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9ba006ae70 0x7f9ba006d330 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.186+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bbc146ba0 0x7f9bbc1324c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.186+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9bbc1467c0 0x7f9bbc0b47d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.186+0000 7f9bc3b01640 1 --2- 172.21.15.72:0/4075705155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bbc06c7e0 0x7f9bbc1f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.186+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 >> 172.21.15.72:0/4075705155 conn(0x7f9bbc0b0a90 msgr2=0x7f9bbc15de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.186+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 shutdown_connections 2023-12-12T09:52:43.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.186+0000 7f9bc3b01640 1 -- 172.21.15.72:0/4075705155 wait complete. 2023-12-12T09:52:43.206 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-12T09:52:41.556619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3196,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3272704,"data_stored":1985289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004621"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070257+0000","last_change":"2023-12-12T09:52:07.005949+0000","last_active":"2023-12-12T09:52:12.070257+0000","last_peered":"2023-12-12T09:52:12.070257+0000","last_clean":"2023-12-12T09:52:12.070257+0000","last_became_active":"2023-12-12T09:52:07.005677+0000","last_became_peered":"2023-12-12T09:52:07.005677+0000","last_unstale":"2023-12-12T09:52:12.070257+0000","last_undegraded":"2023-12-12T09:52:12.070257+0000","last_fullsized":"2023-12-12T09:52:12.070257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:31.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.058906+0000","last_change":"2023-12-12T09:52:08.029768+0000","last_active":"2023-12-12T09:52:12.058906+0000","last_peered":"2023-12-12T09:52:12.058906+0000","last_clean":"2023-12-12T09:52:12.058906+0000","last_became_active":"2023-12-12T09:52:08.029493+0000","last_became_peered":"2023-12-12T09:52:08.029493+0000","last_unstale":"2023-12-12T09:52:12.058906+0000","last_undegraded":"2023-12-12T09:52:12.058906+0000","last_fullsized":"2023-12-12T09:52:12.058906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283436+0000","last_change":"2023-12-12T09:52:12.106967+0000","last_active":"2023-12-12T09:52:13.283436+0000","last_peered":"2023-12-12T09:52:13.283436+0000","last_clean":"2023-12-12T09:52:13.283436+0000","last_became_active":"2023-12-12T09:52:12.106583+0000","last_became_peered":"2023-12-12T09:52:12.106583+0000","last_unstale":"2023-12-12T09:52:13.283436+0000","last_undegraded":"2023-12-12T09:52:13.283436+0000","last_fullsized":"2023-12-12T09:52:13.283436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:02.585452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059029+0000","last_change":"2023-12-12T09:52:06.990826+0000","last_active":"2023-12-12T09:52:12.059029+0000","last_peered":"2023-12-12T09:52:12.059029+0000","last_clean":"2023-12-12T09:52:12.059029+0000","last_became_active":"2023-12-12T09:52:06.990515+0000","last_became_peered":"2023-12-12T09:52:06.990515+0000","last_unstale":"2023-12-12T09:52:12.059029+0000","last_undegraded":"2023-12-12T09:52:12.059029+0000","last_fullsized":"2023-12-12T09:52:12.059029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:19.282948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070081+0000","last_change":"2023-12-12T09:52:08.023719+0000","last_active":"2023-12-12T09:52:12.070081+0000","last_peered":"2023-12-12T09:52:12.070081+0000","last_clean":"2023-12-12T09:52:12.070081+0000","last_became_active":"2023-12-12T09:52:08.023372+0000","last_became_peered":"2023-12-12T09:52:08.023372+0000","last_unstale":"2023-12-12T09:52:12.070081+0000","last_undegraded":"2023-12-12T09:52:12.070081+0000","last_fullsized":"2023-12-12T09:52:12.070081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281424+0000","last_change":"2023-12-12T09:52:12.082594+0000","last_active":"2023-12-12T09:52:13.281424+0000","last_peered":"2023-12-12T09:52:13.281424+0000","last_clean":"2023-12-12T09:52:13.281424+0000","last_became_active":"2023-12-12T09:52:12.081951+0000","last_became_peered":"2023-12-12T09:52:12.081951+0000","last_unstale":"2023-12-12T09:52:13.281424+0000","last_undegraded":"2023-12-12T09:52:13.281424+0000","last_fullsized":"2023-12-12T09:52:13.281424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:45.079238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104327+0000","last_change":"2023-12-12T09:52:07.014029+0000","last_active":"2023-12-12T09:52:12.104327+0000","last_peered":"2023-12-12T09:52:12.104327+0000","last_clean":"2023-12-12T09:52:12.104327+0000","last_became_active":"2023-12-12T09:52:07.013726+0000","last_became_peered":"2023-12-12T09:52:07.013726+0000","last_unstale":"2023-12-12T09:52:12.104327+0000","last_undegraded":"2023-12-12T09:52:12.104327+0000","last_fullsized":"2023-12-12T09:52:12.104327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:46.546348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074859+0000","last_change":"2023-12-12T09:52:08.024974+0000","last_active":"2023-12-12T09:52:12.074859+0000","last_peered":"2023-12-12T09:52:12.074859+0000","last_clean":"2023-12-12T09:52:12.074859+0000","last_became_active":"2023-12-12T09:52:08.024739+0000","last_became_peered":"2023-12-12T09:52:08.024739+0000","last_unstale":"2023-12-12T09:52:12.074859+0000","last_undegraded":"2023-12-12T09:52:12.074859+0000","last_fullsized":"2023-12-12T09:52:12.074859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.950798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281819+0000","last_change":"2023-12-12T09:52:12.082902+0000","last_active":"2023-12-12T09:52:13.281819+0000","last_peered":"2023-12-12T09:52:13.281819+0000","last_clean":"2023-12-12T09:52:13.281819+0000","last_became_active":"2023-12-12T09:52:12.082411+0000","last_became_peered":"2023-12-12T09:52:12.082411+0000","last_unstale":"2023-12-12T09:52:13.281819+0000","last_undegraded":"2023-12-12T09:52:13.281819+0000","last_fullsized":"2023-12-12T09:52:13.281819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:39.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104053+0000","last_change":"2023-12-12T09:52:07.014406+0000","last_active":"2023-12-12T09:52:12.104053+0000","last_peered":"2023-12-12T09:52:12.104053+0000","last_clean":"2023-12-12T09:52:12.104053+0000","last_became_active":"2023-12-12T09:52:07.014067+0000","last_became_peered":"2023-12-12T09:52:07.014067+0000","last_unstale":"2023-12-12T09:52:12.104053+0000","last_undegraded":"2023-12-12T09:52:12.104053+0000","last_fullsized":"2023-12-12T09:52:12.104053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075313+0000","last_change":"2023-12-12T09:52:08.025152+0000","last_active":"2023-12-12T09:52:12.075313+0000","last_peered":"2023-12-12T09:52:12.075313+0000","last_clean":"2023-12-12T09:52:12.075313+0000","last_became_active":"2023-12-12T09:52:08.024282+0000","last_became_peered":"2023-12-12T09:52:08.024282+0000","last_unstale":"2023-12-12T09:52:12.075313+0000","last_undegraded":"2023-12-12T09:52:12.075313+0000","last_fullsized":"2023-12-12T09:52:12.075313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:35.813548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281955+0000","last_change":"2023-12-12T09:52:12.079381+0000","last_active":"2023-12-12T09:52:13.281955+0000","last_peered":"2023-12-12T09:52:13.281955+0000","last_clean":"2023-12-12T09:52:13.281955+0000","last_became_active":"2023-12-12T09:52:12.078805+0000","last_became_peered":"2023-12-12T09:52:12.078805+0000","last_unstale":"2023-12-12T09:52:13.281955+0000","last_undegraded":"2023-12-12T09:52:13.281955+0000","last_fullsized":"2023-12-12T09:52:13.281955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:21:02.014413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050630+0000","last_change":"2023-12-12T09:52:08.020343+0000","last_active":"2023-12-12T09:52:12.050630+0000","last_peered":"2023-12-12T09:52:12.050630+0000","last_clean":"2023-12-12T09:52:12.050630+0000","last_became_active":"2023-12-12T09:52:08.019981+0000","last_became_peered":"2023-12-12T09:52:08.019981+0000","last_unstale":"2023-12-12T09:52:12.050630+0000","last_undegraded":"2023-12-12T09:52:12.050630+0000","last_fullsized":"2023-12-12T09:52:12.050630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104677+0000","last_change":"2023-12-12T09:52:07.011835+0000","last_active":"2023-12-12T09:52:12.104677+0000","last_peered":"2023-12-12T09:52:12.104677+0000","last_clean":"2023-12-12T09:52:12.104677+0000","last_became_active":"2023-12-12T09:52:07.011505+0000","last_became_peered":"2023-12-12T09:52:07.011505+0000","last_unstale":"2023-12-12T09:52:12.104677+0000","last_undegraded":"2023-12-12T09:52:12.104677+0000","last_fullsized":"2023-12-12T09:52:12.104677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285005+0000","last_change":"2023-12-12T09:52:12.080462+0000","last_active":"2023-12-12T09:52:13.285005+0000","last_peered":"2023-12-12T09:52:13.285005+0000","last_clean":"2023-12-12T09:52:13.285005+0000","last_became_active":"2023-12-12T09:52:12.080128+0000","last_became_peered":"2023-12-12T09:52:12.080128+0000","last_unstale":"2023-12-12T09:52:13.285005+0000","last_undegraded":"2023-12-12T09:52:13.285005+0000","last_fullsized":"2023-12-12T09:52:13.285005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054727+0000","last_change":"2023-12-12T09:52:08.028847+0000","last_active":"2023-12-12T09:52:12.054727+0000","last_peered":"2023-12-12T09:52:12.054727+0000","last_clean":"2023-12-12T09:52:12.054727+0000","last_became_active":"2023-12-12T09:52:08.028537+0000","last_became_peered":"2023-12-12T09:52:08.028537+0000","last_unstale":"2023-12-12T09:52:12.054727+0000","last_undegraded":"2023-12-12T09:52:12.054727+0000","last_fullsized":"2023-12-12T09:52:12.054727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068568+0000","last_change":"2023-12-12T09:52:06.992502+0000","last_active":"2023-12-12T09:52:12.068568+0000","last_peered":"2023-12-12T09:52:12.068568+0000","last_clean":"2023-12-12T09:52:12.068568+0000","last_became_active":"2023-12-12T09:52:06.992218+0000","last_became_peered":"2023-12-12T09:52:06.992218+0000","last_unstale":"2023-12-12T09:52:12.068568+0000","last_undegraded":"2023-12-12T09:52:12.068568+0000","last_fullsized":"2023-12-12T09:52:12.068568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284506+0000","last_change":"2023-12-12T09:52:12.083926+0000","last_active":"2023-12-12T09:52:13.284506+0000","last_peered":"2023-12-12T09:52:13.284506+0000","last_clean":"2023-12-12T09:52:13.284506+0000","last_became_active":"2023-12-12T09:52:12.083322+0000","last_became_peered":"2023-12-12T09:52:12.083322+0000","last_unstale":"2023-12-12T09:52:13.284506+0000","last_undegraded":"2023-12-12T09:52:13.284506+0000","last_fullsized":"2023-12-12T09:52:13.284506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:31.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059740+0000","last_change":"2023-12-12T09:52:08.021868+0000","last_active":"2023-12-12T09:52:12.059740+0000","last_peered":"2023-12-12T09:52:12.059740+0000","last_clean":"2023-12-12T09:52:12.059740+0000","last_became_active":"2023-12-12T09:52:08.020477+0000","last_became_peered":"2023-12-12T09:52:08.020477+0000","last_unstale":"2023-12-12T09:52:12.059740+0000","last_undegraded":"2023-12-12T09:52:12.059740+0000","last_fullsized":"2023-12-12T09:52:12.059740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055893+0000","last_change":"2023-12-12T09:52:07.005983+0000","last_active":"2023-12-12T09:52:12.055893+0000","last_peered":"2023-12-12T09:52:12.055893+0000","last_clean":"2023-12-12T09:52:12.055893+0000","last_became_active":"2023-12-12T09:52:07.005723+0000","last_became_peered":"2023-12-12T09:52:07.005723+0000","last_unstale":"2023-12-12T09:52:12.055893+0000","last_undegraded":"2023-12-12T09:52:12.055893+0000","last_fullsized":"2023-12-12T09:52:12.055893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282866+0000","last_change":"2023-12-12T09:52:12.110411+0000","last_active":"2023-12-12T09:52:13.282866+0000","last_peered":"2023-12-12T09:52:13.282866+0000","last_clean":"2023-12-12T09:52:13.282866+0000","last_became_active":"2023-12-12T09:52:12.110045+0000","last_became_peered":"2023-12-12T09:52:12.110045+0000","last_unstale":"2023-12-12T09:52:13.282866+0000","last_undegraded":"2023-12-12T09:52:13.282866+0000","last_fullsized":"2023-12-12T09:52:13.282866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:39.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.069034+0000","last_change":"2023-12-12T09:52:08.034974+0000","last_active":"2023-12-12T09:52:12.069034+0000","last_peered":"2023-12-12T09:52:12.069034+0000","last_clean":"2023-12-12T09:52:12.069034+0000","last_became_active":"2023-12-12T09:52:08.034169+0000","last_became_peered":"2023-12-12T09:52:08.034169+0000","last_unstale":"2023-12-12T09:52:12.069034+0000","last_undegraded":"2023-12-12T09:52:12.069034+0000","last_fullsized":"2023-12-12T09:52:12.069034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:47:41.840110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103698+0000","last_change":"2023-12-12T09:52:07.006046+0000","last_active":"2023-12-12T09:52:12.103698+0000","last_peered":"2023-12-12T09:52:12.103698+0000","last_clean":"2023-12-12T09:52:12.103698+0000","last_became_active":"2023-12-12T09:52:07.005742+0000","last_became_peered":"2023-12-12T09:52:07.005742+0000","last_unstale":"2023-12-12T09:52:12.103698+0000","last_undegraded":"2023-12-12T09:52:12.103698+0000","last_fullsized":"2023-12-12T09:52:12.103698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:31.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285141+0000","last_change":"2023-12-12T09:52:12.084173+0000","last_active":"2023-12-12T09:52:13.285141+0000","last_peered":"2023-12-12T09:52:13.285141+0000","last_clean":"2023-12-12T09:52:13.285141+0000","last_became_active":"2023-12-12T09:52:12.083324+0000","last_became_peered":"2023-12-12T09:52:12.083324+0000","last_unstale":"2023-12-12T09:52:13.285141+0000","last_undegraded":"2023-12-12T09:52:13.285141+0000","last_fullsized":"2023-12-12T09:52:13.285141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.071231+0000","last_change":"2023-12-12T09:52:08.022538+0000","last_active":"2023-12-12T09:52:12.071231+0000","last_peered":"2023-12-12T09:52:12.071231+0000","last_clean":"2023-12-12T09:52:12.071231+0000","last_became_active":"2023-12-12T09:52:08.022274+0000","last_became_peered":"2023-12-12T09:52:08.022274+0000","last_unstale":"2023-12-12T09:52:12.071231+0000","last_undegraded":"2023-12-12T09:52:12.071231+0000","last_fullsized":"2023-12-12T09:52:12.071231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053881+0000","last_change":"2023-12-12T09:52:07.012094+0000","last_active":"2023-12-12T09:52:12.053881+0000","last_peered":"2023-12-12T09:52:12.053881+0000","last_clean":"2023-12-12T09:52:12.053881+0000","last_became_active":"2023-12-12T09:52:07.011693+0000","last_became_peered":"2023-12-12T09:52:07.011693+0000","last_unstale":"2023-12-12T09:52:12.053881+0000","last_undegraded":"2023-12-12T09:52:12.053881+0000","last_fullsized":"2023-12-12T09:52:12.053881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:47.890555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281952+0000","last_change":"2023-12-12T09:52:12.111616+0000","last_active":"2023-12-12T09:52:13.281952+0000","last_peered":"2023-12-12T09:52:13.281952+0000","last_clean":"2023-12-12T09:52:13.281952+0000","last_became_active":"2023-12-12T09:52:12.110326+0000","last_became_peered":"2023-12-12T09:52:12.110326+0000","last_unstale":"2023-12-12T09:52:13.281952+0000","last_undegraded":"2023-12-12T09:52:13.281952+0000","last_fullsized":"2023-12-12T09:52:13.281952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:34:01.832456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.063408+0000","last_change":"2023-12-12T09:52:08.034520+0000","last_active":"2023-12-12T09:52:12.063408+0000","last_peered":"2023-12-12T09:52:12.063408+0000","last_clean":"2023-12-12T09:52:12.063408+0000","last_became_active":"2023-12-12T09:52:08.034232+0000","last_became_peered":"2023-12-12T09:52:08.034232+0000","last_unstale":"2023-12-12T09:52:12.063408+0000","last_undegraded":"2023-12-12T09:52:12.063408+0000","last_fullsized":"2023-12-12T09:52:12.063408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.760548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.057080+0000","last_change":"2023-12-12T09:52:07.004065+0000","last_active":"2023-12-12T09:52:12.057080+0000","last_peered":"2023-12-12T09:52:12.057080+0000","last_clean":"2023-12-12T09:52:12.057080+0000","last_became_active":"2023-12-12T09:52:07.003690+0000","last_became_peered":"2023-12-12T09:52:07.003690+0000","last_unstale":"2023-12-12T09:52:12.057080+0000","last_undegraded":"2023-12-12T09:52:12.057080+0000","last_fullsized":"2023-12-12T09:52:12.057080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:47.890555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285159+0000","last_change":"2023-12-12T09:52:12.108409+0000","last_active":"2023-12-12T09:52:13.285159+0000","last_peered":"2023-12-12T09:52:13.285159+0000","last_clean":"2023-12-12T09:52:13.285159+0000","last_became_active":"2023-12-12T09:52:12.107934+0000","last_became_peered":"2023-12-12T09:52:12.107934+0000","last_unstale":"2023-12-12T09:52:13.285159+0000","last_undegraded":"2023-12-12T09:52:13.285159+0000","last_fullsized":"2023-12-12T09:52:13.285159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:20.552141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065611+0000","last_change":"2023-12-12T09:52:08.031660+0000","last_active":"2023-12-12T09:52:12.065611+0000","last_peered":"2023-12-12T09:52:12.065611+0000","last_clean":"2023-12-12T09:52:12.065611+0000","last_became_active":"2023-12-12T09:52:08.030341+0000","last_became_peered":"2023-12-12T09:52:08.030341+0000","last_unstale":"2023-12-12T09:52:12.065611+0000","last_undegraded":"2023-12-12T09:52:12.065611+0000","last_fullsized":"2023-12-12T09:52:12.065611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104424+0000","last_change":"2023-12-12T09:52:07.014640+0000","last_active":"2023-12-12T09:52:12.104424+0000","last_peered":"2023-12-12T09:52:12.104424+0000","last_clean":"2023-12-12T09:52:12.104424+0000","last_became_active":"2023-12-12T09:52:07.014313+0000","last_became_peered":"2023-12-12T09:52:07.014313+0000","last_unstale":"2023-12-12T09:52:12.104424+0000","last_undegraded":"2023-12-12T09:52:12.104424+0000","last_fullsized":"2023-12-12T09:52:12.104424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:25.514763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285307+0000","last_change":"2023-12-12T09:52:12.111140+0000","last_active":"2023-12-12T09:52:13.285307+0000","last_peered":"2023-12-12T09:52:13.285307+0000","last_clean":"2023-12-12T09:52:13.285307+0000","last_became_active":"2023-12-12T09:52:12.110437+0000","last_became_peered":"2023-12-12T09:52:12.110437+0000","last_unstale":"2023-12-12T09:52:13.285307+0000","last_undegraded":"2023-12-12T09:52:13.285307+0000","last_fullsized":"2023-12-12T09:52:13.285307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075331+0000","last_change":"2023-12-12T09:52:08.031638+0000","last_active":"2023-12-12T09:52:12.075331+0000","last_peered":"2023-12-12T09:52:12.075331+0000","last_clean":"2023-12-12T09:52:12.075331+0000","last_became_active":"2023-12-12T09:52:08.031330+0000","last_became_peered":"2023-12-12T09:52:08.031330+0000","last_unstale":"2023-12-12T09:52:12.075331+0000","last_undegraded":"2023-12-12T09:52:12.075331+0000","last_fullsized":"2023-12-12T09:52:12.075331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:27.371886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065722+0000","last_change":"2023-12-12T09:52:06.996775+0000","last_active":"2023-12-12T09:52:12.065722+0000","last_peered":"2023-12-12T09:52:12.065722+0000","last_clean":"2023-12-12T09:52:12.065722+0000","last_became_active":"2023-12-12T09:52:06.995618+0000","last_became_peered":"2023-12-12T09:52:06.995618+0000","last_unstale":"2023-12-12T09:52:12.065722+0000","last_undegraded":"2023-12-12T09:52:12.065722+0000","last_fullsized":"2023-12-12T09:52:12.065722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:26:34.669025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282930+0000","last_change":"2023-12-12T09:52:12.085685+0000","last_active":"2023-12-12T09:52:13.282930+0000","last_peered":"2023-12-12T09:52:13.282930+0000","last_clean":"2023-12-12T09:52:13.282930+0000","last_became_active":"2023-12-12T09:52:12.085247+0000","last_became_peered":"2023-12-12T09:52:12.085247+0000","last_unstale":"2023-12-12T09:52:13.282930+0000","last_undegraded":"2023-12-12T09:52:13.282930+0000","last_fullsized":"2023-12-12T09:52:13.282930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:31.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070066+0000","last_change":"2023-12-12T09:52:08.020254+0000","last_active":"2023-12-12T09:52:12.070066+0000","last_peered":"2023-12-12T09:52:12.070066+0000","last_clean":"2023-12-12T09:52:12.070066+0000","last_became_active":"2023-12-12T09:52:08.019979+0000","last_became_peered":"2023-12-12T09:52:08.019979+0000","last_unstale":"2023-12-12T09:52:12.070066+0000","last_undegraded":"2023-12-12T09:52:12.070066+0000","last_fullsized":"2023-12-12T09:52:12.070066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:32.163069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074498+0000","last_change":"2023-12-12T09:52:06.997560+0000","last_active":"2023-12-12T09:52:12.074498+0000","last_peered":"2023-12-12T09:52:12.074498+0000","last_clean":"2023-12-12T09:52:12.074498+0000","last_became_active":"2023-12-12T09:52:06.996940+0000","last_became_peered":"2023-12-12T09:52:06.996940+0000","last_unstale":"2023-12-12T09:52:12.074498+0000","last_undegraded":"2023-12-12T09:52:12.074498+0000","last_fullsized":"2023-12-12T09:52:12.074498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:09.895493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285696+0000","last_change":"2023-12-12T09:52:12.077447+0000","last_active":"2023-12-12T09:52:13.285696+0000","last_peered":"2023-12-12T09:52:13.285696+0000","last_clean":"2023-12-12T09:52:13.285696+0000","last_became_active":"2023-12-12T09:52:12.076927+0000","last_became_peered":"2023-12-12T09:52:12.076927+0000","last_unstale":"2023-12-12T09:52:13.285696+0000","last_undegraded":"2023-12-12T09:52:13.285696+0000","last_fullsized":"2023-12-12T09:52:13.285696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:55.497789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.057627+0000","last_change":"2023-12-12T09:52:08.035111+0000","last_active":"2023-12-12T09:52:12.057627+0000","last_peered":"2023-12-12T09:52:12.057627+0000","last_clean":"2023-12-12T09:52:12.057627+0000","last_became_active":"2023-12-12T09:52:08.034823+0000","last_became_peered":"2023-12-12T09:52:08.034823+0000","last_unstale":"2023-12-12T09:52:12.057627+0000","last_undegraded":"2023-12-12T09:52:12.057627+0000","last_fullsized":"2023-12-12T09:52:12.057627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055046+0000","last_change":"2023-12-12T09:52:06.993403+0000","last_active":"2023-12-12T09:52:12.055046+0000","last_peered":"2023-12-12T09:52:12.055046+0000","last_clean":"2023-12-12T09:52:12.055046+0000","last_became_active":"2023-12-12T09:52:06.993116+0000","last_became_peered":"2023-12-12T09:52:06.993116+0000","last_unstale":"2023-12-12T09:52:12.055046+0000","last_undegraded":"2023-12-12T09:52:12.055046+0000","last_fullsized":"2023-12-12T09:52:12.055046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283254+0000","last_change":"2023-12-12T09:52:12.080184+0000","last_active":"2023-12-12T09:52:13.283254+0000","last_peered":"2023-12-12T09:52:13.283254+0000","last_clean":"2023-12-12T09:52:13.283254+0000","last_became_active":"2023-12-12T09:52:12.079804+0000","last_became_peered":"2023-12-12T09:52:12.079804+0000","last_unstale":"2023-12-12T09:52:13.283254+0000","last_undegraded":"2023-12-12T09:52:13.283254+0000","last_fullsized":"2023-12-12T09:52:13.283254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065376+0000","last_change":"2023-12-12T09:52:08.026908+0000","last_active":"2023-12-12T09:52:12.065376+0000","last_peered":"2023-12-12T09:52:12.065376+0000","last_clean":"2023-12-12T09:52:12.065376+0000","last_became_active":"2023-12-12T09:52:08.025912+0000","last_became_peered":"2023-12-12T09:52:08.025912+0000","last_unstale":"2023-12-12T09:52:12.065376+0000","last_undegraded":"2023-12-12T09:52:12.065376+0000","last_fullsized":"2023-12-12T09:52:12.065376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103688+0000","last_change":"2023-12-12T09:52:07.009290+0000","last_active":"2023-12-12T09:52:12.103688+0000","last_peered":"2023-12-12T09:52:12.103688+0000","last_clean":"2023-12-12T09:52:12.103688+0000","last_became_active":"2023-12-12T09:52:07.008896+0000","last_became_peered":"2023-12-12T09:52:07.008896+0000","last_unstale":"2023-12-12T09:52:12.103688+0000","last_undegraded":"2023-12-12T09:52:12.103688+0000","last_fullsized":"2023-12-12T09:52:12.103688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:21.191123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.290045+0000","last_change":"2023-12-12T09:52:12.115265+0000","last_active":"2023-12-12T09:52:13.290045+0000","last_peered":"2023-12-12T09:52:13.290045+0000","last_clean":"2023-12-12T09:52:13.290045+0000","last_became_active":"2023-12-12T09:52:12.114674+0000","last_became_peered":"2023-12-12T09:52:12.114674+0000","last_unstale":"2023-12-12T09:52:13.290045+0000","last_undegraded":"2023-12-12T09:52:13.290045+0000","last_fullsized":"2023-12-12T09:52:13.290045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:34:54.772153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060809+0000","last_change":"2023-12-12T09:52:08.028299+0000","last_active":"2023-12-12T09:52:12.060809+0000","last_peered":"2023-12-12T09:52:12.060809+0000","last_clean":"2023-12-12T09:52:12.060809+0000","last_became_active":"2023-12-12T09:52:08.028080+0000","last_became_peered":"2023-12-12T09:52:08.028080+0000","last_unstale":"2023-12-12T09:52:12.060809+0000","last_undegraded":"2023-12-12T09:52:12.060809+0000","last_fullsized":"2023-12-12T09:52:12.060809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074571+0000","last_change":"2023-12-12T09:52:06.996346+0000","last_active":"2023-12-12T09:52:12.074571+0000","last_peered":"2023-12-12T09:52:12.074571+0000","last_clean":"2023-12-12T09:52:12.074571+0000","last_became_active":"2023-12-12T09:52:06.995462+0000","last_became_peered":"2023-12-12T09:52:06.995462+0000","last_unstale":"2023-12-12T09:52:12.074571+0000","last_undegraded":"2023-12-12T09:52:12.074571+0000","last_fullsized":"2023-12-12T09:52:12.074571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:59.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103796+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:52:12.103796+0000","last_peered":"2023-12-12T09:52:12.103796+0000","last_clean":"2023-12-12T09:52:12.103796+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:52:12.103796+0000","last_undegraded":"2023-12-12T09:52:12.103796+0000","last_fullsized":"2023-12-12T09:52:12.103796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.055422+0000","last_change":"2023-12-12T09:52:12.109683+0000","last_active":"2023-12-12T09:52:13.055422+0000","last_peered":"2023-12-12T09:52:13.055422+0000","last_clean":"2023-12-12T09:52:13.055422+0000","last_became_active":"2023-12-12T09:52:12.108319+0000","last_became_peered":"2023-12-12T09:52:12.108319+0000","last_unstale":"2023-12-12T09:52:13.055422+0000","last_undegraded":"2023-12-12T09:52:13.055422+0000","last_fullsized":"2023-12-12T09:52:13.055422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:34:01.832456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104240+0000","last_change":"2023-12-12T09:52:08.033828+0000","last_active":"2023-12-12T09:52:12.104240+0000","last_peered":"2023-12-12T09:52:12.104240+0000","last_clean":"2023-12-12T09:52:12.104240+0000","last_became_active":"2023-12-12T09:52:08.030396+0000","last_became_peered":"2023-12-12T09:52:08.030396+0000","last_unstale":"2023-12-12T09:52:12.104240+0000","last_undegraded":"2023-12-12T09:52:12.104240+0000","last_fullsized":"2023-12-12T09:52:12.104240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:27.371886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060933+0000","last_change":"2023-12-12T09:52:06.998515+0000","last_active":"2023-12-12T09:52:12.060933+0000","last_peered":"2023-12-12T09:52:12.060933+0000","last_clean":"2023-12-12T09:52:12.060933+0000","last_became_active":"2023-12-12T09:52:06.998220+0000","last_became_peered":"2023-12-12T09:52:06.998220+0000","last_unstale":"2023-12-12T09:52:12.060933+0000","last_undegraded":"2023-12-12T09:52:12.060933+0000","last_fullsized":"2023-12-12T09:52:12.060933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:19.282948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.054359+0000","last_change":"2023-12-12T09:52:12.106813+0000","last_active":"2023-12-12T09:52:13.054359+0000","last_peered":"2023-12-12T09:52:13.054359+0000","last_clean":"2023-12-12T09:52:13.054359+0000","last_became_active":"2023-12-12T09:52:12.106334+0000","last_became_peered":"2023-12-12T09:52:12.106334+0000","last_unstale":"2023-12-12T09:52:13.054359+0000","last_undegraded":"2023-12-12T09:52:13.054359+0000","last_fullsized":"2023-12-12T09:52:13.054359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:39.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074947+0000","last_change":"2023-12-12T09:52:08.033424+0000","last_active":"2023-12-12T09:52:12.074947+0000","last_peered":"2023-12-12T09:52:12.074947+0000","last_clean":"2023-12-12T09:52:12.074947+0000","last_became_active":"2023-12-12T09:52:08.030055+0000","last_became_peered":"2023-12-12T09:52:08.030055+0000","last_unstale":"2023-12-12T09:52:12.074947+0000","last_undegraded":"2023-12-12T09:52:12.074947+0000","last_fullsized":"2023-12-12T09:52:12.074947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:32.450565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056464+0000","last_change":"2023-12-12T09:52:07.000843+0000","last_active":"2023-12-12T09:52:12.056464+0000","last_peered":"2023-12-12T09:52:12.056464+0000","last_clean":"2023-12-12T09:52:12.056464+0000","last_became_active":"2023-12-12T09:52:07.000476+0000","last_became_peered":"2023-12-12T09:52:07.000476+0000","last_unstale":"2023-12-12T09:52:12.056464+0000","last_undegraded":"2023-12-12T09:52:12.056464+0000","last_fullsized":"2023-12-12T09:52:12.056464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:59.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285509+0000","last_change":"2023-12-12T09:52:12.082760+0000","last_active":"2023-12-12T09:52:13.285509+0000","last_peered":"2023-12-12T09:52:13.285509+0000","last_clean":"2023-12-12T09:52:13.285509+0000","last_became_active":"2023-12-12T09:52:12.082269+0000","last_became_peered":"2023-12-12T09:52:12.082269+0000","last_unstale":"2023-12-12T09:52:13.285509+0000","last_undegraded":"2023-12-12T09:52:13.285509+0000","last_fullsized":"2023-12-12T09:52:13.285509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104860+0000","last_change":"2023-12-12T09:52:08.035636+0000","last_active":"2023-12-12T09:52:12.104860+0000","last_peered":"2023-12-12T09:52:12.104860+0000","last_clean":"2023-12-12T09:52:12.104860+0000","last_became_active":"2023-12-12T09:52:08.035306+0000","last_became_peered":"2023-12-12T09:52:08.035306+0000","last_unstale":"2023-12-12T09:52:12.104860+0000","last_undegraded":"2023-12-12T09:52:12.104860+0000","last_fullsized":"2023-12-12T09:52:12.104860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:34:50.708538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053859+0000","last_change":"2023-12-12T09:52:06.993300+0000","last_active":"2023-12-12T09:52:12.053859+0000","last_peered":"2023-12-12T09:52:12.053859+0000","last_clean":"2023-12-12T09:52:12.053859+0000","last_became_active":"2023-12-12T09:52:06.992970+0000","last_became_peered":"2023-12-12T09:52:06.992970+0000","last_unstale":"2023-12-12T09:52:12.053859+0000","last_undegraded":"2023-12-12T09:52:12.053859+0000","last_fullsized":"2023-12-12T09:52:12.053859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:21.191123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.057884+0000","last_change":"2023-12-12T09:52:12.107302+0000","last_active":"2023-12-12T09:52:13.057884+0000","last_peered":"2023-12-12T09:52:13.057884+0000","last_clean":"2023-12-12T09:52:13.057884+0000","last_became_active":"2023-12-12T09:52:12.106810+0000","last_became_peered":"2023-12-12T09:52:12.106810+0000","last_unstale":"2023-12-12T09:52:13.057884+0000","last_undegraded":"2023-12-12T09:52:13.057884+0000","last_fullsized":"2023-12-12T09:52:13.057884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:25:03.440488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104689+0000","last_change":"2023-12-12T09:52:08.039951+0000","last_active":"2023-12-12T09:52:12.104689+0000","last_peered":"2023-12-12T09:52:12.104689+0000","last_clean":"2023-12-12T09:52:12.104689+0000","last_became_active":"2023-12-12T09:52:08.039124+0000","last_became_peered":"2023-12-12T09:52:08.039124+0000","last_unstale":"2023-12-12T09:52:12.104689+0000","last_undegraded":"2023-12-12T09:52:12.104689+0000","last_fullsized":"2023-12-12T09:52:12.104689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:32.450565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066373+0000","last_change":"2023-12-12T09:52:06.998901+0000","last_active":"2023-12-12T09:52:12.066373+0000","last_peered":"2023-12-12T09:52:12.066373+0000","last_clean":"2023-12-12T09:52:12.066373+0000","last_became_active":"2023-12-12T09:52:06.998458+0000","last_became_peered":"2023-12-12T09:52:06.998458+0000","last_unstale":"2023-12-12T09:52:12.066373+0000","last_undegraded":"2023-12-12T09:52:12.066373+0000","last_fullsized":"2023-12-12T09:52:12.066373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281456+0000","last_change":"2023-12-12T09:52:12.080392+0000","last_active":"2023-12-12T09:52:13.281456+0000","last_peered":"2023-12-12T09:52:13.281456+0000","last_clean":"2023-12-12T09:52:13.281456+0000","last_became_active":"2023-12-12T09:52:12.079941+0000","last_became_peered":"2023-12-12T09:52:12.079941+0000","last_unstale":"2023-12-12T09:52:13.281456+0000","last_undegraded":"2023-12-12T09:52:13.281456+0000","last_fullsized":"2023-12-12T09:52:13.281456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:10.824163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103824+0000","last_change":"2023-12-12T09:52:08.032605+0000","last_active":"2023-12-12T09:52:12.103824+0000","last_peered":"2023-12-12T09:52:12.103824+0000","last_clean":"2023-12-12T09:52:12.103824+0000","last_became_active":"2023-12-12T09:52:08.032321+0000","last_became_peered":"2023-12-12T09:52:08.032321+0000","last_unstale":"2023-12-12T09:52:12.103824+0000","last_undegraded":"2023-12-12T09:52:12.103824+0000","last_fullsized":"2023-12-12T09:52:12.103824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053766+0000","last_change":"2023-12-12T09:52:06.993710+0000","last_active":"2023-12-12T09:52:12.053766+0000","last_peered":"2023-12-12T09:52:12.053766+0000","last_clean":"2023-12-12T09:52:12.053766+0000","last_became_active":"2023-12-12T09:52:06.993427+0000","last_became_peered":"2023-12-12T09:52:06.993427+0000","last_unstale":"2023-12-12T09:52:12.053766+0000","last_undegraded":"2023-12-12T09:52:12.053766+0000","last_fullsized":"2023-12-12T09:52:12.053766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:39.005895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.289994+0000","last_change":"2023-12-12T09:52:12.083094+0000","last_active":"2023-12-12T09:52:13.289994+0000","last_peered":"2023-12-12T09:52:13.289994+0000","last_clean":"2023-12-12T09:52:13.289994+0000","last_became_active":"2023-12-12T09:52:12.082050+0000","last_became_peered":"2023-12-12T09:52:12.082050+0000","last_unstale":"2023-12-12T09:52:13.289994+0000","last_undegraded":"2023-12-12T09:52:13.289994+0000","last_fullsized":"2023-12-12T09:52:13.289994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:51.971159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050599+0000","last_change":"2023-12-12T09:52:08.020100+0000","last_active":"2023-12-12T09:52:12.050599+0000","last_peered":"2023-12-12T09:52:12.050599+0000","last_clean":"2023-12-12T09:52:12.050599+0000","last_became_active":"2023-12-12T09:52:08.019776+0000","last_became_peered":"2023-12-12T09:52:08.019776+0000","last_unstale":"2023-12-12T09:52:12.050599+0000","last_undegraded":"2023-12-12T09:52:12.050599+0000","last_fullsized":"2023-12-12T09:52:12.050599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050514+0000","last_change":"2023-12-12T09:52:07.002330+0000","last_active":"2023-12-12T09:52:12.050514+0000","last_peered":"2023-12-12T09:52:12.050514+0000","last_clean":"2023-12-12T09:52:12.050514+0000","last_became_active":"2023-12-12T09:52:07.001838+0000","last_became_peered":"2023-12-12T09:52:07.001838+0000","last_unstale":"2023-12-12T09:52:12.050514+0000","last_undegraded":"2023-12-12T09:52:12.050514+0000","last_fullsized":"2023-12-12T09:52:12.050514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:44.065611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.287475+0000","last_change":"2023-12-12T09:52:12.111595+0000","last_active":"2023-12-12T09:52:13.287475+0000","last_peered":"2023-12-12T09:52:13.287475+0000","last_clean":"2023-12-12T09:52:13.287475+0000","last_became_active":"2023-12-12T09:52:12.111143+0000","last_became_peered":"2023-12-12T09:52:12.111143+0000","last_unstale":"2023-12-12T09:52:13.287475+0000","last_undegraded":"2023-12-12T09:52:13.287475+0000","last_fullsized":"2023-12-12T09:52:13.287475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:07:22.383529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104101+0000","last_change":"2023-12-12T09:52:08.036655+0000","last_active":"2023-12-12T09:52:12.104101+0000","last_peered":"2023-12-12T09:52:12.104101+0000","last_clean":"2023-12-12T09:52:12.104101+0000","last_became_active":"2023-12-12T09:52:08.036260+0000","last_became_peered":"2023-12-12T09:52:08.036260+0000","last_unstale":"2023-12-12T09:52:12.104101+0000","last_undegraded":"2023-12-12T09:52:12.104101+0000","last_fullsized":"2023-12-12T09:52:12.104101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:35.813548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074437+0000","last_change":"2023-12-12T09:52:06.999233+0000","last_active":"2023-12-12T09:52:12.074437+0000","last_peered":"2023-12-12T09:52:12.074437+0000","last_clean":"2023-12-12T09:52:12.074437+0000","last_became_active":"2023-12-12T09:52:06.998962+0000","last_became_peered":"2023-12-12T09:52:06.998962+0000","last_unstale":"2023-12-12T09:52:12.074437+0000","last_undegraded":"2023-12-12T09:52:12.074437+0000","last_fullsized":"2023-12-12T09:52:12.074437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:25.514763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285663+0000","last_change":"2023-12-12T09:52:12.106941+0000","last_active":"2023-12-12T09:52:13.285663+0000","last_peered":"2023-12-12T09:52:13.285663+0000","last_clean":"2023-12-12T09:52:13.285663+0000","last_became_active":"2023-12-12T09:52:12.106547+0000","last_became_peered":"2023-12-12T09:52:12.106547+0000","last_unstale":"2023-12-12T09:52:13.285663+0000","last_undegraded":"2023-12-12T09:52:13.285663+0000","last_fullsized":"2023-12-12T09:52:13.285663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:04:27.791759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070910+0000","last_change":"2023-12-12T09:52:08.029830+0000","last_active":"2023-12-12T09:52:12.070910+0000","last_peered":"2023-12-12T09:52:12.070910+0000","last_clean":"2023-12-12T09:52:12.070910+0000","last_became_active":"2023-12-12T09:52:08.029455+0000","last_became_peered":"2023-12-12T09:52:08.029455+0000","last_unstale":"2023-12-12T09:52:12.070910+0000","last_undegraded":"2023-12-12T09:52:12.070910+0000","last_fullsized":"2023-12-12T09:52:12.070910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068345+0000","last_change":"2023-12-12T09:52:07.000933+0000","last_active":"2023-12-12T09:52:12.068345+0000","last_peered":"2023-12-12T09:52:12.068345+0000","last_clean":"2023-12-12T09:52:12.068345+0000","last_became_active":"2023-12-12T09:52:07.000646+0000","last_became_peered":"2023-12-12T09:52:07.000646+0000","last_unstale":"2023-12-12T09:52:12.068345+0000","last_undegraded":"2023-12-12T09:52:12.068345+0000","last_fullsized":"2023-12-12T09:52:12.068345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:44.065611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.059459+0000","last_change":"2023-12-12T09:52:12.110657+0000","last_active":"2023-12-12T09:52:13.059459+0000","last_peered":"2023-12-12T09:52:13.059459+0000","last_clean":"2023-12-12T09:52:13.059459+0000","last_became_active":"2023-12-12T09:52:12.109951+0000","last_became_peered":"2023-12-12T09:52:12.109951+0000","last_unstale":"2023-12-12T09:52:13.059459+0000","last_undegraded":"2023-12-12T09:52:13.059459+0000","last_fullsized":"2023-12-12T09:52:13.059459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:20.552141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104135+0000","last_change":"2023-12-12T09:52:08.035067+0000","last_active":"2023-12-12T09:52:12.104135+0000","last_peered":"2023-12-12T09:52:12.104135+0000","last_clean":"2023-12-12T09:52:12.104135+0000","last_became_active":"2023-12-12T09:52:08.034803+0000","last_became_peered":"2023-12-12T09:52:08.034803+0000","last_unstale":"2023-12-12T09:52:12.104135+0000","last_undegraded":"2023-12-12T09:52:12.104135+0000","last_fullsized":"2023-12-12T09:52:12.104135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:22.978364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:27.255184+0000","last_change":"2023-12-12T09:52:06.998675+0000","last_active":"2023-12-12T09:52:27.255184+0000","last_peered":"2023-12-12T09:52:27.255184+0000","last_clean":"2023-12-12T09:52:27.255184+0000","last_became_active":"2023-12-12T09:52:06.998364+0000","last_became_peered":"2023-12-12T09:52:06.998364+0000","last_unstale":"2023-12-12T09:52:27.255184+0000","last_undegraded":"2023-12-12T09:52:27.255184+0000","last_fullsized":"2023-12-12T09:52:27.255184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3839,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283450+0000","last_change":"2023-12-12T09:52:12.086437+0000","last_active":"2023-12-12T09:52:13.283450+0000","last_peered":"2023-12-12T09:52:13.283450+0000","last_clean":"2023-12-12T09:52:13.283450+0000","last_became_active":"2023-12-12T09:52:12.083068+0000","last_became_peered":"2023-12-12T09:52:12.083068+0000","last_unstale":"2023-12-12T09:52:13.283450+0000","last_undegraded":"2023-12-12T09:52:13.283450+0000","last_fullsized":"2023-12-12T09:52:13.283450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066756+0000","last_change":"2023-12-12T09:52:08.030198+0000","last_active":"2023-12-12T09:52:12.066756+0000","last_peered":"2023-12-12T09:52:12.066756+0000","last_clean":"2023-12-12T09:52:12.066756+0000","last_became_active":"2023-12-12T09:52:08.027733+0000","last_became_peered":"2023-12-12T09:52:08.027733+0000","last_unstale":"2023-12-12T09:52:12.066756+0000","last_undegraded":"2023-12-12T09:52:12.066756+0000","last_fullsized":"2023-12-12T09:52:12.066756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:08.788149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056359+0000","last_change":"2023-12-12T09:52:07.004177+0000","last_active":"2023-12-12T09:52:12.056359+0000","last_peered":"2023-12-12T09:52:12.056359+0000","last_clean":"2023-12-12T09:52:12.056359+0000","last_became_active":"2023-12-12T09:52:07.003836+0000","last_became_peered":"2023-12-12T09:52:07.003836+0000","last_unstale":"2023-12-12T09:52:12.056359+0000","last_undegraded":"2023-12-12T09:52:12.056359+0000","last_fullsized":"2023-12-12T09:52:12.056359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284483+0000","last_change":"2023-12-12T09:52:12.081391+0000","last_active":"2023-12-12T09:52:13.284483+0000","last_peered":"2023-12-12T09:52:13.284483+0000","last_clean":"2023-12-12T09:52:13.284483+0000","last_became_active":"2023-12-12T09:52:12.080949+0000","last_became_peered":"2023-12-12T09:52:12.080949+0000","last_unstale":"2023-12-12T09:52:13.284483+0000","last_undegraded":"2023-12-12T09:52:13.284483+0000","last_fullsized":"2023-12-12T09:52:13.284483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:39.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070164+0000","last_change":"2023-12-12T09:52:08.023308+0000","last_active":"2023-12-12T09:52:12.070164+0000","last_peered":"2023-12-12T09:52:12.070164+0000","last_clean":"2023-12-12T09:52:12.070164+0000","last_became_active":"2023-12-12T09:52:08.023048+0000","last_became_peered":"2023-12-12T09:52:08.023048+0000","last_unstale":"2023-12-12T09:52:12.070164+0000","last_undegraded":"2023-12-12T09:52:12.070164+0000","last_fullsized":"2023-12-12T09:52:12.070164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075599+0000","last_change":"2023-12-12T09:52:07.001746+0000","last_active":"2023-12-12T09:52:12.075599+0000","last_peered":"2023-12-12T09:52:12.075599+0000","last_clean":"2023-12-12T09:52:12.075599+0000","last_became_active":"2023-12-12T09:52:07.001424+0000","last_became_peered":"2023-12-12T09:52:07.001424+0000","last_unstale":"2023-12-12T09:52:12.075599+0000","last_undegraded":"2023-12-12T09:52:12.075599+0000","last_fullsized":"2023-12-12T09:52:12.075599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:54.297647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282877+0000","last_change":"2023-12-12T09:52:12.077880+0000","last_active":"2023-12-12T09:52:13.282877+0000","last_peered":"2023-12-12T09:52:13.282877+0000","last_clean":"2023-12-12T09:52:13.282877+0000","last_became_active":"2023-12-12T09:52:12.077365+0000","last_became_peered":"2023-12-12T09:52:12.077365+0000","last_unstale":"2023-12-12T09:52:13.282877+0000","last_undegraded":"2023-12-12T09:52:13.282877+0000","last_fullsized":"2023-12-12T09:52:13.282877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:43:05.307563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075525+0000","last_change":"2023-12-12T09:52:08.029359+0000","last_active":"2023-12-12T09:52:12.075525+0000","last_peered":"2023-12-12T09:52:12.075525+0000","last_clean":"2023-12-12T09:52:12.075525+0000","last_became_active":"2023-12-12T09:52:08.028997+0000","last_became_peered":"2023-12-12T09:52:08.028997+0000","last_unstale":"2023-12-12T09:52:12.075525+0000","last_undegraded":"2023-12-12T09:52:12.075525+0000","last_fullsized":"2023-12-12T09:52:12.075525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:22.978364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.052142+0000","last_change":"2023-12-12T09:52:07.006953+0000","last_active":"2023-12-12T09:52:12.052142+0000","last_peered":"2023-12-12T09:52:12.052142+0000","last_clean":"2023-12-12T09:52:12.052142+0000","last_became_active":"2023-12-12T09:52:07.006666+0000","last_became_peered":"2023-12-12T09:52:07.006666+0000","last_unstale":"2023-12-12T09:52:12.052142+0000","last_undegraded":"2023-12-12T09:52:12.052142+0000","last_fullsized":"2023-12-12T09:52:12.052142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:26:34.669025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283756+0000","last_change":"2023-12-12T09:52:12.104433+0000","last_active":"2023-12-12T09:52:13.283756+0000","last_peered":"2023-12-12T09:52:13.283756+0000","last_clean":"2023-12-12T09:52:13.283756+0000","last_became_active":"2023-12-12T09:52:12.104126+0000","last_became_peered":"2023-12-12T09:52:12.104126+0000","last_unstale":"2023-12-12T09:52:13.283756+0000","last_undegraded":"2023-12-12T09:52:13.283756+0000","last_fullsized":"2023-12-12T09:52:13.283756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:25:03.440488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.289874+0000","last_change":"2023-12-12T09:52:12.111376+0000","last_active":"2023-12-12T09:52:13.289874+0000","last_peered":"2023-12-12T09:52:13.289874+0000","last_clean":"2023-12-12T09:52:13.289874+0000","last_became_active":"2023-12-12T09:52:12.110863+0000","last_became_peered":"2023-12-12T09:52:12.110863+0000","last_unstale":"2023-12-12T09:52:13.289874+0000","last_undegraded":"2023-12-12T09:52:13.289874+0000","last_fullsized":"2023-12-12T09:52:13.289874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:55.497789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055671+0000","last_change":"2023-12-12T09:52:08.024511+0000","last_active":"2023-12-12T09:52:12.055671+0000","last_peered":"2023-12-12T09:52:12.055671+0000","last_clean":"2023-12-12T09:52:12.055671+0000","last_became_active":"2023-12-12T09:52:08.024258+0000","last_became_peered":"2023-12-12T09:52:08.024258+0000","last_unstale":"2023-12-12T09:52:12.055671+0000","last_undegraded":"2023-12-12T09:52:12.055671+0000","last_fullsized":"2023-12-12T09:52:12.055671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074250+0000","last_change":"2023-12-12T09:52:07.000641+0000","last_active":"2023-12-12T09:52:12.074250+0000","last_peered":"2023-12-12T09:52:12.074250+0000","last_clean":"2023-12-12T09:52:12.074250+0000","last_became_active":"2023-12-12T09:52:07.000245+0000","last_became_peered":"2023-12-12T09:52:07.000245+0000","last_unstale":"2023-12-12T09:52:12.074250+0000","last_undegraded":"2023-12-12T09:52:12.074250+0000","last_fullsized":"2023-12-12T09:52:12.074250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:46.546348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.056067+0000","last_change":"2023-12-12T09:52:12.078229+0000","last_active":"2023-12-12T09:52:13.056067+0000","last_peered":"2023-12-12T09:52:13.056067+0000","last_clean":"2023-12-12T09:52:13.056067+0000","last_became_active":"2023-12-12T09:52:12.077764+0000","last_became_peered":"2023-12-12T09:52:12.077764+0000","last_unstale":"2023-12-12T09:52:13.056067+0000","last_undegraded":"2023-12-12T09:52:13.056067+0000","last_fullsized":"2023-12-12T09:52:13.056067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:21:02.014413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059639+0000","last_change":"2023-12-12T09:52:08.028827+0000","last_active":"2023-12-12T09:52:12.059639+0000","last_peered":"2023-12-12T09:52:12.059639+0000","last_clean":"2023-12-12T09:52:12.059639+0000","last_became_active":"2023-12-12T09:52:08.024860+0000","last_became_peered":"2023-12-12T09:52:08.024860+0000","last_unstale":"2023-12-12T09:52:12.059639+0000","last_undegraded":"2023-12-12T09:52:12.059639+0000","last_fullsized":"2023-12-12T09:52:12.059639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054975+0000","last_change":"2023-12-12T09:52:06.999381+0000","last_active":"2023-12-12T09:52:12.054975+0000","last_peered":"2023-12-12T09:52:12.054975+0000","last_clean":"2023-12-12T09:52:12.054975+0000","last_became_active":"2023-12-12T09:52:06.999099+0000","last_became_peered":"2023-12-12T09:52:06.999099+0000","last_unstale":"2023-12-12T09:52:12.054975+0000","last_undegraded":"2023-12-12T09:52:12.054975+0000","last_fullsized":"2023-12-12T09:52:12.054975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050899+0000","last_change":"2023-12-12T09:52:07.003879+0000","last_active":"2023-12-12T09:52:12.050899+0000","last_peered":"2023-12-12T09:52:12.050899+0000","last_clean":"2023-12-12T09:52:12.050899+0000","last_became_active":"2023-12-12T09:52:07.002760+0000","last_became_peered":"2023-12-12T09:52:07.002760+0000","last_unstale":"2023-12-12T09:52:12.050899+0000","last_undegraded":"2023-12-12T09:52:12.050899+0000","last_fullsized":"2023-12-12T09:52:12.050899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:07.777932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068247+0000","last_change":"2023-12-12T09:52:08.026740+0000","last_active":"2023-12-12T09:52:12.068247+0000","last_peered":"2023-12-12T09:52:12.068247+0000","last_clean":"2023-12-12T09:52:12.068247+0000","last_became_active":"2023-12-12T09:52:08.024698+0000","last_became_peered":"2023-12-12T09:52:08.024698+0000","last_unstale":"2023-12-12T09:52:12.068247+0000","last_undegraded":"2023-12-12T09:52:12.068247+0000","last_fullsized":"2023-12-12T09:52:12.068247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284952+0000","last_change":"2023-12-12T09:52:12.078748+0000","last_active":"2023-12-12T09:52:13.284952+0000","last_peered":"2023-12-12T09:52:13.284952+0000","last_clean":"2023-12-12T09:52:13.284952+0000","last_became_active":"2023-12-12T09:52:12.077889+0000","last_became_peered":"2023-12-12T09:52:12.077889+0000","last_unstale":"2023-12-12T09:52:13.284952+0000","last_undegraded":"2023-12-12T09:52:13.284952+0000","last_fullsized":"2023-12-12T09:52:13.284952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:05.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054823+0000","last_change":"2023-12-12T09:52:07.008787+0000","last_active":"2023-12-12T09:52:12.054823+0000","last_peered":"2023-12-12T09:52:12.054823+0000","last_clean":"2023-12-12T09:52:12.054823+0000","last_became_active":"2023-12-12T09:52:07.008455+0000","last_became_peered":"2023-12-12T09:52:07.008455+0000","last_unstale":"2023-12-12T09:52:12.054823+0000","last_undegraded":"2023-12-12T09:52:12.054823+0000","last_fullsized":"2023-12-12T09:52:12.054823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.061074+0000","last_change":"2023-12-12T09:52:08.022255+0000","last_active":"2023-12-12T09:52:12.061074+0000","last_peered":"2023-12-12T09:52:12.061074+0000","last_clean":"2023-12-12T09:52:12.061074+0000","last_became_active":"2023-12-12T09:52:08.020240+0000","last_became_peered":"2023-12-12T09:52:08.020240+0000","last_unstale":"2023-12-12T09:52:12.061074+0000","last_undegraded":"2023-12-12T09:52:12.061074+0000","last_fullsized":"2023-12-12T09:52:12.061074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:22.201340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.286721+0000","last_change":"2023-12-12T09:52:12.081878+0000","last_active":"2023-12-12T09:52:13.286721+0000","last_peered":"2023-12-12T09:52:13.286721+0000","last_clean":"2023-12-12T09:52:13.286721+0000","last_became_active":"2023-12-12T09:52:12.081502+0000","last_became_peered":"2023-12-12T09:52:12.081502+0000","last_unstale":"2023-12-12T09:52:13.286721+0000","last_undegraded":"2023-12-12T09:52:13.286721+0000","last_fullsized":"2023-12-12T09:52:13.286721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:18:41.559574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050317+0000","last_change":"2023-12-12T09:52:08.021237+0000","last_active":"2023-12-12T09:52:12.050317+0000","last_peered":"2023-12-12T09:52:12.050317+0000","last_clean":"2023-12-12T09:52:12.050317+0000","last_became_active":"2023-12-12T09:52:08.019519+0000","last_became_peered":"2023-12-12T09:52:08.019519+0000","last_unstale":"2023-12-12T09:52:12.050317+0000","last_undegraded":"2023-12-12T09:52:12.050317+0000","last_fullsized":"2023-12-12T09:52:12.050317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:01.082803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054863+0000","last_change":"2023-12-12T09:52:08.019363+0000","last_active":"2023-12-12T09:52:12.054863+0000","last_peered":"2023-12-12T09:52:12.054863+0000","last_clean":"2023-12-12T09:52:12.054863+0000","last_became_active":"2023-12-12T09:52:08.012574+0000","last_became_peered":"2023-12-12T09:52:08.012574+0000","last_unstale":"2023-12-12T09:52:12.054863+0000","last_undegraded":"2023-12-12T09:52:12.054863+0000","last_fullsized":"2023-12-12T09:52:12.054863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:08.788149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074382+0000","last_change":"2023-12-12T09:52:08.030165+0000","last_active":"2023-12-12T09:52:12.074382+0000","last_peered":"2023-12-12T09:52:12.074382+0000","last_clean":"2023-12-12T09:52:12.074382+0000","last_became_active":"2023-12-12T09:52:08.029626+0000","last_became_peered":"2023-12-12T09:52:08.029626+0000","last_unstale":"2023-12-12T09:52:12.074382+0000","last_undegraded":"2023-12-12T09:52:12.074382+0000","last_fullsized":"2023-12-12T09:52:12.074382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068155+0000","last_change":"2023-12-12T09:52:08.040599+0000","last_active":"2023-12-12T09:52:12.068155+0000","last_peered":"2023-12-12T09:52:12.068155+0000","last_clean":"2023-12-12T09:52:12.068155+0000","last_became_active":"2023-12-12T09:52:08.039875+0000","last_became_peered":"2023-12-12T09:52:08.039875+0000","last_unstale":"2023-12-12T09:52:12.068155+0000","last_undegraded":"2023-12-12T09:52:12.068155+0000","last_fullsized":"2023-12-12T09:52:12.068155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060010+0000","last_change":"2023-12-12T09:52:08.032398+0000","last_active":"2023-12-12T09:52:12.060010+0000","last_peered":"2023-12-12T09:52:12.060010+0000","last_clean":"2023-12-12T09:52:12.060010+0000","last_became_active":"2023-12-12T09:52:08.031194+0000","last_became_peered":"2023-12-12T09:52:08.031194+0000","last_unstale":"2023-12-12T09:52:12.060010+0000","last_undegraded":"2023-12-12T09:52:12.060010+0000","last_fullsized":"2023-12-12T09:52:12.060010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056859+0000","last_change":"2023-12-12T09:52:08.023831+0000","last_active":"2023-12-12T09:52:12.056859+0000","last_peered":"2023-12-12T09:52:12.056859+0000","last_clean":"2023-12-12T09:52:12.056859+0000","last_became_active":"2023-12-12T09:52:08.023532+0000","last_became_peered":"2023-12-12T09:52:08.023532+0000","last_unstale":"2023-12-12T09:52:12.056859+0000","last_undegraded":"2023-12-12T09:52:12.056859+0000","last_fullsized":"2023-12-12T09:52:12.056859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054670+0000","last_change":"2023-12-12T09:52:08.028457+0000","last_active":"2023-12-12T09:52:12.054670+0000","last_peered":"2023-12-12T09:52:12.054670+0000","last_clean":"2023-12-12T09:52:12.054670+0000","last_became_active":"2023-12-12T09:52:08.028172+0000","last_became_peered":"2023-12-12T09:52:08.028172+0000","last_unstale":"2023-12-12T09:52:12.054670+0000","last_undegraded":"2023-12-12T09:52:12.054670+0000","last_fullsized":"2023-12-12T09:52:12.054670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.063395+0000","last_change":"2023-12-12T09:52:08.033841+0000","last_active":"2023-12-12T09:52:12.063395+0000","last_peered":"2023-12-12T09:52:12.063395+0000","last_clean":"2023-12-12T09:52:12.063395+0000","last_became_active":"2023-12-12T09:52:08.030880+0000","last_became_peered":"2023-12-12T09:52:08.030880+0000","last_unstale":"2023-12-12T09:52:12.063395+0000","last_undegraded":"2023-12-12T09:52:12.063395+0000","last_fullsized":"2023-12-12T09:52:12.063395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074138+0000","last_change":"2023-12-12T09:52:08.032051+0000","last_active":"2023-12-12T09:52:12.074138+0000","last_peered":"2023-12-12T09:52:12.074138+0000","last_clean":"2023-12-12T09:52:12.074138+0000","last_became_active":"2023-12-12T09:52:08.031785+0000","last_became_peered":"2023-12-12T09:52:08.031785+0000","last_unstale":"2023-12-12T09:52:12.074138+0000","last_undegraded":"2023-12-12T09:52:12.074138+0000","last_fullsized":"2023-12-12T09:52:12.074138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054886+0000","last_change":"2023-12-12T09:52:08.031522+0000","last_active":"2023-12-12T09:52:12.054886+0000","last_peered":"2023-12-12T09:52:12.054886+0000","last_clean":"2023-12-12T09:52:12.054886+0000","last_became_active":"2023-12-12T09:52:08.031262+0000","last_became_peered":"2023-12-12T09:52:08.031262+0000","last_unstale":"2023-12-12T09:52:12.054886+0000","last_undegraded":"2023-12-12T09:52:12.054886+0000","last_fullsized":"2023-12-12T09:52:12.054886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074154+0000","last_change":"2023-12-12T09:52:08.024395+0000","last_active":"2023-12-12T09:52:12.074154+0000","last_peered":"2023-12-12T09:52:12.074154+0000","last_clean":"2023-12-12T09:52:12.074154+0000","last_became_active":"2023-12-12T09:52:08.024105+0000","last_became_peered":"2023-12-12T09:52:08.024105+0000","last_unstale":"2023-12-12T09:52:12.074154+0000","last_undegraded":"2023-12-12T09:52:12.074154+0000","last_fullsized":"2023-12-12T09:52:12.074154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.760548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.069490+0000","last_change":"2023-12-12T09:52:08.037425+0000","last_active":"2023-12-12T09:52:12.069490+0000","last_peered":"2023-12-12T09:52:12.069490+0000","last_clean":"2023-12-12T09:52:12.069490+0000","last_became_active":"2023-12-12T09:52:08.035756+0000","last_became_peered":"2023-12-12T09:52:08.035756+0000","last_unstale":"2023-12-12T09:52:12.069490+0000","last_undegraded":"2023-12-12T09:52:12.069490+0000","last_fullsized":"2023-12-12T09:52:12.069490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:01.082803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050903+0000","last_change":"2023-12-12T09:52:08.031988+0000","last_active":"2023-12-12T09:52:12.050903+0000","last_peered":"2023-12-12T09:52:12.050903+0000","last_clean":"2023-12-12T09:52:12.050903+0000","last_became_active":"2023-12-12T09:52:08.031525+0000","last_became_peered":"2023-12-12T09:52:08.031525+0000","last_unstale":"2023-12-12T09:52:12.050903+0000","last_undegraded":"2023-12-12T09:52:12.050903+0000","last_fullsized":"2023-12-12T09:52:12.050903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104392+0000","last_change":"2023-12-12T09:52:08.034992+0000","last_active":"2023-12-12T09:52:12.104392+0000","last_peered":"2023-12-12T09:52:12.104392+0000","last_clean":"2023-12-12T09:52:12.104392+0000","last_became_active":"2023-12-12T09:52:08.034424+0000","last_became_peered":"2023-12-12T09:52:08.034424+0000","last_unstale":"2023-12-12T09:52:12.104392+0000","last_undegraded":"2023-12-12T09:52:12.104392+0000","last_fullsized":"2023-12-12T09:52:12.104392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:24.493525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053756+0000","last_change":"2023-12-12T09:52:08.021603+0000","last_active":"2023-12-12T09:52:12.053756+0000","last_peered":"2023-12-12T09:52:12.053756+0000","last_clean":"2023-12-12T09:52:12.053756+0000","last_became_active":"2023-12-12T09:52:08.021253+0000","last_became_peered":"2023-12-12T09:52:08.021253+0000","last_unstale":"2023-12-12T09:52:12.053756+0000","last_undegraded":"2023-12-12T09:52:12.053756+0000","last_fullsized":"2023-12-12T09:52:12.053756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:45.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075822+0000","last_change":"2023-12-12T09:52:08.036026+0000","last_active":"2023-12-12T09:52:12.075822+0000","last_peered":"2023-12-12T09:52:12.075822+0000","last_clean":"2023-12-12T09:52:12.075822+0000","last_became_active":"2023-12-12T09:52:08.035390+0000","last_became_peered":"2023-12-12T09:52:08.035390+0000","last_unstale":"2023-12-12T09:52:12.075822+0000","last_undegraded":"2023-12-12T09:52:12.075822+0000","last_fullsized":"2023-12-12T09:52:12.075822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:51.434174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.073460+0000","last_change":"2023-12-12T09:52:08.013438+0000","last_active":"2023-12-12T09:52:12.073460+0000","last_peered":"2023-12-12T09:52:12.073460+0000","last_clean":"2023-12-12T09:52:12.073460+0000","last_became_active":"2023-12-12T09:52:08.013002+0000","last_became_peered":"2023-12-12T09:52:08.013002+0000","last_unstale":"2023-12-12T09:52:12.073460+0000","last_undegraded":"2023-12-12T09:52:12.073460+0000","last_fullsized":"2023-12-12T09:52:12.073460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055788+0000","last_change":"2023-12-12T09:52:08.032162+0000","last_active":"2023-12-12T09:52:12.055788+0000","last_peered":"2023-12-12T09:52:12.055788+0000","last_clean":"2023-12-12T09:52:12.055788+0000","last_became_active":"2023-12-12T09:52:08.031735+0000","last_became_peered":"2023-12-12T09:52:08.031735+0000","last_unstale":"2023-12-12T09:52:12.055788+0000","last_undegraded":"2023-12-12T09:52:12.055788+0000","last_fullsized":"2023-12-12T09:52:12.055788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070823+0000","last_change":"2023-12-12T09:52:08.024576+0000","last_active":"2023-12-12T09:52:12.070823+0000","last_peered":"2023-12-12T09:52:12.070823+0000","last_clean":"2023-12-12T09:52:12.070823+0000","last_became_active":"2023-12-12T09:52:08.024277+0000","last_became_peered":"2023-12-12T09:52:08.024277+0000","last_unstale":"2023-12-12T09:52:12.070823+0000","last_undegraded":"2023-12-12T09:52:12.070823+0000","last_fullsized":"2023-12-12T09:52:12.070823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:48.900772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054635+0000","last_change":"2023-12-12T09:52:08.029262+0000","last_active":"2023-12-12T09:52:12.054635+0000","last_peered":"2023-12-12T09:52:12.054635+0000","last_clean":"2023-12-12T09:52:12.054635+0000","last_became_active":"2023-12-12T09:52:08.028949+0000","last_became_peered":"2023-12-12T09:52:08.028949+0000","last_unstale":"2023-12-12T09:52:12.054635+0000","last_undegraded":"2023-12-12T09:52:12.054635+0000","last_fullsized":"2023-12-12T09:52:12.054635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066542+0000","last_change":"2023-12-12T09:52:08.024637+0000","last_active":"2023-12-12T09:52:12.066542+0000","last_peered":"2023-12-12T09:52:12.066542+0000","last_clean":"2023-12-12T09:52:12.066542+0000","last_became_active":"2023-12-12T09:52:08.023754+0000","last_became_peered":"2023-12-12T09:52:08.023754+0000","last_unstale":"2023-12-12T09:52:12.066542+0000","last_undegraded":"2023-12-12T09:52:12.066542+0000","last_fullsized":"2023-12-12T09:52:12.066542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:45.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055913+0000","last_change":"2023-12-12T09:52:08.024613+0000","last_active":"2023-12-12T09:52:12.055913+0000","last_peered":"2023-12-12T09:52:12.055913+0000","last_clean":"2023-12-12T09:52:12.055913+0000","last_became_active":"2023-12-12T09:52:08.024257+0000","last_became_peered":"2023-12-12T09:52:08.024257+0000","last_unstale":"2023-12-12T09:52:12.055913+0000","last_undegraded":"2023-12-12T09:52:12.055913+0000","last_fullsized":"2023-12-12T09:52:12.055913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054417+0000","last_change":"2023-12-12T09:52:08.023097+0000","last_active":"2023-12-12T09:52:12.054417+0000","last_peered":"2023-12-12T09:52:12.054417+0000","last_clean":"2023-12-12T09:52:12.054417+0000","last_became_active":"2023-12-12T09:52:08.022866+0000","last_became_peered":"2023-12-12T09:52:08.022866+0000","last_unstale":"2023-12-12T09:52:12.054417+0000","last_undegraded":"2023-12-12T09:52:12.054417+0000","last_fullsized":"2023-12-12T09:52:12.054417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104813+0000","last_change":"2023-12-12T09:52:08.039672+0000","last_active":"2023-12-12T09:52:12.104813+0000","last_peered":"2023-12-12T09:52:12.104813+0000","last_clean":"2023-12-12T09:52:12.104813+0000","last_became_active":"2023-12-12T09:52:08.039145+0000","last_became_peered":"2023-12-12T09:52:08.039145+0000","last_unstale":"2023-12-12T09:52:12.104813+0000","last_undegraded":"2023-12-12T09:52:12.104813+0000","last_fullsized":"2023-12-12T09:52:12.104813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:47.907544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059506+0000","last_change":"2023-12-12T09:52:08.030292+0000","last_active":"2023-12-12T09:52:12.059506+0000","last_peered":"2023-12-12T09:52:12.059506+0000","last_clean":"2023-12-12T09:52:12.059506+0000","last_became_active":"2023-12-12T09:52:08.029117+0000","last_became_peered":"2023-12-12T09:52:08.029117+0000","last_unstale":"2023-12-12T09:52:12.059506+0000","last_undegraded":"2023-12-12T09:52:12.059506+0000","last_fullsized":"2023-12-12T09:52:12.059506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103702+0000","last_change":"2023-12-12T09:52:08.040587+0000","last_active":"2023-12-12T09:52:12.103702+0000","last_peered":"2023-12-12T09:52:12.103702+0000","last_clean":"2023-12-12T09:52:12.103702+0000","last_became_active":"2023-12-12T09:52:08.040204+0000","last_became_peered":"2023-12-12T09:52:08.040204+0000","last_unstale":"2023-12-12T09:52:12.103702+0000","last_undegraded":"2023-12-12T09:52:12.103702+0000","last_fullsized":"2023-12-12T09:52:12.103702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.073636+0000","last_change":"2023-12-12T09:52:08.015126+0000","last_active":"2023-12-12T09:52:12.073636+0000","last_peered":"2023-12-12T09:52:12.073636+0000","last_clean":"2023-12-12T09:52:12.073636+0000","last_became_active":"2023-12-12T09:52:08.014817+0000","last_became_peered":"2023-12-12T09:52:08.014817+0000","last_unstale":"2023-12-12T09:52:12.073636+0000","last_undegraded":"2023-12-12T09:52:12.073636+0000","last_fullsized":"2023-12-12T09:52:12.073636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103966+0000","last_change":"2023-12-12T09:52:08.034136+0000","last_active":"2023-12-12T09:52:12.103966+0000","last_peered":"2023-12-12T09:52:12.103966+0000","last_clean":"2023-12-12T09:52:12.103966+0000","last_became_active":"2023-12-12T09:52:08.033810+0000","last_became_peered":"2023-12-12T09:52:08.033810+0000","last_unstale":"2023-12-12T09:52:12.103966+0000","last_undegraded":"2023-12-12T09:52:12.103966+0000","last_fullsized":"2023-12-12T09:52:12.103966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.950798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074787+0000","last_change":"2023-12-12T09:52:08.032774+0000","last_active":"2023-12-12T09:52:12.074787+0000","last_peered":"2023-12-12T09:52:12.074787+0000","last_clean":"2023-12-12T09:52:12.074787+0000","last_became_active":"2023-12-12T09:52:08.030758+0000","last_became_peered":"2023-12-12T09:52:08.030758+0000","last_unstale":"2023-12-12T09:52:12.074787+0000","last_undegraded":"2023-12-12T09:52:12.074787+0000","last_fullsized":"2023-12-12T09:52:12.074787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065632+0000","last_change":"2023-12-12T09:52:08.032385+0000","last_active":"2023-12-12T09:52:12.065632+0000","last_peered":"2023-12-12T09:52:12.065632+0000","last_clean":"2023-12-12T09:52:12.065632+0000","last_became_active":"2023-12-12T09:52:08.032067+0000","last_became_peered":"2023-12-12T09:52:08.032067+0000","last_unstale":"2023-12-12T09:52:12.065632+0000","last_undegraded":"2023-12-12T09:52:12.065632+0000","last_fullsized":"2023-12-12T09:52:12.065632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056995+0000","last_change":"2023-12-12T09:52:08.028105+0000","last_active":"2023-12-12T09:52:12.056995+0000","last_peered":"2023-12-12T09:52:12.056995+0000","last_clean":"2023-12-12T09:52:12.056995+0000","last_became_active":"2023-12-12T09:52:08.027786+0000","last_became_peered":"2023-12-12T09:52:08.027786+0000","last_unstale":"2023-12-12T09:52:12.056995+0000","last_undegraded":"2023-12-12T09:52:12.056995+0000","last_fullsized":"2023-12-12T09:52:12.056995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104765+0000","last_change":"2023-12-12T09:52:08.034061+0000","last_active":"2023-12-12T09:52:12.104765+0000","last_peered":"2023-12-12T09:52:12.104765+0000","last_clean":"2023-12-12T09:52:12.104765+0000","last_became_active":"2023-12-12T09:52:08.033745+0000","last_became_peered":"2023-12-12T09:52:08.033745+0000","last_unstale":"2023-12-12T09:52:12.104765+0000","last_undegraded":"2023-12-12T09:52:12.104765+0000","last_fullsized":"2023-12-12T09:52:12.104765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:51.434174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4637,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7970000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2749999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7669999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.821}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8720000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":98899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5840000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":98933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8279999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8690000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.738}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.823}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.653}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:52:43.211 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:52:43.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.457+0000 7f1f67a90640 1 Processor -- start 2023-12-12T09:52:43.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.457+0000 7f1f67a90640 1 -- start start 2023-12-12T09:52:43.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.459+0000 7f1f67a90640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1f600cd000 0x7f1f6006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.459+0000 7f1f67a90640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6006d510 con 0x7f1f601467c0 2023-12-12T09:52:43.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.459+0000 7f1f67a90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6006d7f0 con 0x7f1f60146ba0 2023-12-12T09:52:43.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.459+0000 7f1f67a90640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600ac860 con 0x7f1f600cd000 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.459+0000 7f1f6502b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1f600cd000 0x7f1f6006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.459+0000 7f1f6502b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1f600cd000 0x7f1f6006c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40650/0 (socket says 172.21.15.72:40650) 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.459+0000 7f1f6502b640 1 -- 172.21.15.72:0/2968037612 learned_addr learned my addr 172.21.15.72:0/2968037612 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.459+0000 7f1f6582c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1f601467c0 0x7f1f600cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52490/0 (socket says 172.21.15.72:52490) 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.460+0000 7f1f6502b640 1 -- 172.21.15.72:0/2968037612 >> v1:172.21.15.113:6789/0 conn(0x7f1f60146ba0 legacy=0x7f1f6006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.460+0000 7f1f6502b640 1 -- 172.21.15.72:0/2968037612 >> v1:172.21.15.72:6789/0 conn(0x7f1f601467c0 legacy=0x7f1f600cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.460+0000 7f1f6502b640 1 -- 172.21.15.72:0/2968037612 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6006dad0 con 0x7f1f600cd000 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.461+0000 7f1f6502b640 1 --2- 172.21.15.72:0/2968037612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f600cd000 0x7f1f6006c7e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009870 tx=0x7f1f5002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c587659e622694c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.461+0000 7f1f6482a640 1 -- 172.21.15.72:0/2968037612 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50030860 con 0x7f1f600cd000 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.461+0000 7f1f6482a640 1 -- 172.21.15.72:0/2968037612 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f500309c0 con 0x7f1f600cd000 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.461+0000 7f1f6482a640 1 -- 172.21.15.72:0/2968037612 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50030d50 con 0x7f1f600cd000 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.462+0000 7f1f67a90640 1 -- 172.21.15.72:0/2968037612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f600cd000 msgr2=0x7f1f6006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.462+0000 7f1f67a90640 1 --2- 172.21.15.72:0/2968037612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f600cd000 0x7f1f6006c7e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009870 tx=0x7f1f5002b9a0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.462+0000 7f1f67a90640 1 -- 172.21.15.72:0/2968037612 shutdown_connections 2023-12-12T09:52:43.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.462+0000 7f1f67a90640 1 --2- 172.21.15.72:0/2968037612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f600cd000 0x7f1f6006c7e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.462+0000 7f1f67a90640 1 -- 172.21.15.72:0/2968037612 >> 172.21.15.72:0/2968037612 conn(0x7f1f600b0a90 msgr2=0x7f1f600b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:43.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.462+0000 7f1f67a90640 1 -- 172.21.15.72:0/2968037612 shutdown_connections 2023-12-12T09:52:43.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.462+0000 7f1f67a90640 1 -- 172.21.15.72:0/2968037612 wait complete. 2023-12-12T09:52:43.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.464+0000 7f1f67a90640 1 Processor -- start 2023-12-12T09:52:43.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.464+0000 7f1f67a90640 1 -- start start 2023-12-12T09:52:43.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f67a90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f600cd000 0x7f1f6015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f67a90640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f601467c0 0x7f1f601e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f67a90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f60146ba0 0x7f1f601e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f67a90640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60157ed0 con 0x7f1f601467c0 2023-12-12T09:52:43.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f67a90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60158140 con 0x7f1f600cd000 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f67a90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60158420 con 0x7f1f60146ba0 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6502b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f601467c0 0x7f1f601e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6502b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f601467c0 0x7f1f601e7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40662/0 (socket says 172.21.15.72:40662) 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6502b640 1 -- 172.21.15.72:0/123253632 learned_addr learned my addr 172.21.15.72:0/123253632 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6582c640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f600cd000 0x7f1f6015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6602d640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f60146ba0 0x7f1f601e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6502b640 1 -- 172.21.15.72:0/123253632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f60146ba0 msgr2=0x7f1f601e81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6502b640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f60146ba0 0x7f1f601e81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6502b640 1 -- 172.21.15.72:0/123253632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f600cd000 msgr2=0x7f1f6015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6502b640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f600cd000 0x7f1f6015d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.465+0000 7f1f6502b640 1 -- 172.21.15.72:0/123253632 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500094d0 con 0x7f1f601467c0 2023-12-12T09:52:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.466+0000 7f1f6502b640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f601467c0 0x7f1f601e7c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009870 tx=0x7f1f50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.466+0000 7f1f567fc640 1 -- 172.21.15.72:0/123253632 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50030810 con 0x7f1f601467c0 2023-12-12T09:52:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.466+0000 7f1f567fc640 1 -- 172.21.15.72:0/123253632 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f50043400 con 0x7f1f601467c0 2023-12-12T09:52:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.466+0000 7f1f567fc640 1 -- 172.21.15.72:0/123253632 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f500437b0 con 0x7f1f601467c0 2023-12-12T09:52:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.466+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f60132760 con 0x7f1f601467c0 2023-12-12T09:52:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.466+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f600b26f0 con 0x7f1f601467c0 2023-12-12T09:52:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.467+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f600745d0 con 0x7f1f601467c0 2023-12-12T09:52:43.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.468+0000 7f1f567fc640 1 -- 172.21.15.72:0/123253632 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f50043910 con 0x7f1f601467c0 2023-12-12T09:52:43.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.469+0000 7f1f567fc640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1f4406ae40 0x7f1f4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.469+0000 7f1f567fc640 1 -- 172.21.15.72:0/123253632 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1f5003e070 con 0x7f1f601467c0 2023-12-12T09:52:43.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.469+0000 7f1f6582c640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1f4406ae40 0x7f1f4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.470+0000 7f1f6582c640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1f4406ae40 0x7f1f4406d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c007fd0 tx=0x7f1f5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:43.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.473+0000 7f1f567fc640 1 -- 172.21.15.72:0/123253632 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f50077b20 con 0x7f1f601467c0 2023-12-12T09:52:43.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.647+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f600ae380 con 0x7f1f4406ae40 2023-12-12T09:52:43.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.656+0000 7f1f567fc640 1 -- 172.21.15.72:0/123253632 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345123 (secure 0 0 0) 0x7f1f600ae380 con 0x7f1f4406ae40 2023-12-12T09:52:43.663 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:43.665 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:52:43.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.661+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1f4406ae40 msgr2=0x7f1f4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.661+0000 7f1f67a90640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1f4406ae40 0x7f1f4406d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c007fd0 tx=0x7f1f5c009040 comp rx=0 tx=0).stop 2023-12-12T09:52:43.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.661+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f601467c0 msgr2=0x7f1f601e7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.661+0000 7f1f67a90640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f601467c0 0x7f1f601e7c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009870 tx=0x7f1f50004730 comp rx=0 tx=0).stop 2023-12-12T09:52:43.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.662+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 shutdown_connections 2023-12-12T09:52:43.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.662+0000 7f1f67a90640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1f4406ae40 0x7f1f4406d300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.662+0000 7f1f67a90640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f60146ba0 0x7f1f601e81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.662+0000 7f1f67a90640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f601467c0 0x7f1f601e7c60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.662+0000 7f1f67a90640 1 --2- 172.21.15.72:0/123253632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f600cd000 0x7f1f6015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.662+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 >> 172.21.15.72:0/123253632 conn(0x7f1f600b0a90 msgr2=0x7f1f6015e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:43.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.662+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 shutdown_connections 2023-12-12T09:52:43.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.662+0000 7f1f67a90640 1 -- 172.21.15.72:0/123253632 wait complete. 2023-12-12T09:52:43.682 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-12-12T09:52:43.557773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3196,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3272704,"data_stored":1985289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005113"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070257+0000","last_change":"2023-12-12T09:52:07.005949+0000","last_active":"2023-12-12T09:52:12.070257+0000","last_peered":"2023-12-12T09:52:12.070257+0000","last_clean":"2023-12-12T09:52:12.070257+0000","last_became_active":"2023-12-12T09:52:07.005677+0000","last_became_peered":"2023-12-12T09:52:07.005677+0000","last_unstale":"2023-12-12T09:52:12.070257+0000","last_undegraded":"2023-12-12T09:52:12.070257+0000","last_fullsized":"2023-12-12T09:52:12.070257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:31.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.058906+0000","last_change":"2023-12-12T09:52:08.029768+0000","last_active":"2023-12-12T09:52:12.058906+0000","last_peered":"2023-12-12T09:52:12.058906+0000","last_clean":"2023-12-12T09:52:12.058906+0000","last_became_active":"2023-12-12T09:52:08.029493+0000","last_became_peered":"2023-12-12T09:52:08.029493+0000","last_unstale":"2023-12-12T09:52:12.058906+0000","last_undegraded":"2023-12-12T09:52:12.058906+0000","last_fullsized":"2023-12-12T09:52:12.058906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283436+0000","last_change":"2023-12-12T09:52:12.106967+0000","last_active":"2023-12-12T09:52:13.283436+0000","last_peered":"2023-12-12T09:52:13.283436+0000","last_clean":"2023-12-12T09:52:13.283436+0000","last_became_active":"2023-12-12T09:52:12.106583+0000","last_became_peered":"2023-12-12T09:52:12.106583+0000","last_unstale":"2023-12-12T09:52:13.283436+0000","last_undegraded":"2023-12-12T09:52:13.283436+0000","last_fullsized":"2023-12-12T09:52:13.283436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:02.585452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059029+0000","last_change":"2023-12-12T09:52:06.990826+0000","last_active":"2023-12-12T09:52:12.059029+0000","last_peered":"2023-12-12T09:52:12.059029+0000","last_clean":"2023-12-12T09:52:12.059029+0000","last_became_active":"2023-12-12T09:52:06.990515+0000","last_became_peered":"2023-12-12T09:52:06.990515+0000","last_unstale":"2023-12-12T09:52:12.059029+0000","last_undegraded":"2023-12-12T09:52:12.059029+0000","last_fullsized":"2023-12-12T09:52:12.059029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:19.282948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070081+0000","last_change":"2023-12-12T09:52:08.023719+0000","last_active":"2023-12-12T09:52:12.070081+0000","last_peered":"2023-12-12T09:52:12.070081+0000","last_clean":"2023-12-12T09:52:12.070081+0000","last_became_active":"2023-12-12T09:52:08.023372+0000","last_became_peered":"2023-12-12T09:52:08.023372+0000","last_unstale":"2023-12-12T09:52:12.070081+0000","last_undegraded":"2023-12-12T09:52:12.070081+0000","last_fullsized":"2023-12-12T09:52:12.070081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281424+0000","last_change":"2023-12-12T09:52:12.082594+0000","last_active":"2023-12-12T09:52:13.281424+0000","last_peered":"2023-12-12T09:52:13.281424+0000","last_clean":"2023-12-12T09:52:13.281424+0000","last_became_active":"2023-12-12T09:52:12.081951+0000","last_became_peered":"2023-12-12T09:52:12.081951+0000","last_unstale":"2023-12-12T09:52:13.281424+0000","last_undegraded":"2023-12-12T09:52:13.281424+0000","last_fullsized":"2023-12-12T09:52:13.281424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:45.079238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104327+0000","last_change":"2023-12-12T09:52:07.014029+0000","last_active":"2023-12-12T09:52:12.104327+0000","last_peered":"2023-12-12T09:52:12.104327+0000","last_clean":"2023-12-12T09:52:12.104327+0000","last_became_active":"2023-12-12T09:52:07.013726+0000","last_became_peered":"2023-12-12T09:52:07.013726+0000","last_unstale":"2023-12-12T09:52:12.104327+0000","last_undegraded":"2023-12-12T09:52:12.104327+0000","last_fullsized":"2023-12-12T09:52:12.104327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:46.546348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074859+0000","last_change":"2023-12-12T09:52:08.024974+0000","last_active":"2023-12-12T09:52:12.074859+0000","last_peered":"2023-12-12T09:52:12.074859+0000","last_clean":"2023-12-12T09:52:12.074859+0000","last_became_active":"2023-12-12T09:52:08.024739+0000","last_became_peered":"2023-12-12T09:52:08.024739+0000","last_unstale":"2023-12-12T09:52:12.074859+0000","last_undegraded":"2023-12-12T09:52:12.074859+0000","last_fullsized":"2023-12-12T09:52:12.074859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.950798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281819+0000","last_change":"2023-12-12T09:52:12.082902+0000","last_active":"2023-12-12T09:52:13.281819+0000","last_peered":"2023-12-12T09:52:13.281819+0000","last_clean":"2023-12-12T09:52:13.281819+0000","last_became_active":"2023-12-12T09:52:12.082411+0000","last_became_peered":"2023-12-12T09:52:12.082411+0000","last_unstale":"2023-12-12T09:52:13.281819+0000","last_undegraded":"2023-12-12T09:52:13.281819+0000","last_fullsized":"2023-12-12T09:52:13.281819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:39.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104053+0000","last_change":"2023-12-12T09:52:07.014406+0000","last_active":"2023-12-12T09:52:12.104053+0000","last_peered":"2023-12-12T09:52:12.104053+0000","last_clean":"2023-12-12T09:52:12.104053+0000","last_became_active":"2023-12-12T09:52:07.014067+0000","last_became_peered":"2023-12-12T09:52:07.014067+0000","last_unstale":"2023-12-12T09:52:12.104053+0000","last_undegraded":"2023-12-12T09:52:12.104053+0000","last_fullsized":"2023-12-12T09:52:12.104053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075313+0000","last_change":"2023-12-12T09:52:08.025152+0000","last_active":"2023-12-12T09:52:12.075313+0000","last_peered":"2023-12-12T09:52:12.075313+0000","last_clean":"2023-12-12T09:52:12.075313+0000","last_became_active":"2023-12-12T09:52:08.024282+0000","last_became_peered":"2023-12-12T09:52:08.024282+0000","last_unstale":"2023-12-12T09:52:12.075313+0000","last_undegraded":"2023-12-12T09:52:12.075313+0000","last_fullsized":"2023-12-12T09:52:12.075313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:35.813548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281955+0000","last_change":"2023-12-12T09:52:12.079381+0000","last_active":"2023-12-12T09:52:13.281955+0000","last_peered":"2023-12-12T09:52:13.281955+0000","last_clean":"2023-12-12T09:52:13.281955+0000","last_became_active":"2023-12-12T09:52:12.078805+0000","last_became_peered":"2023-12-12T09:52:12.078805+0000","last_unstale":"2023-12-12T09:52:13.281955+0000","last_undegraded":"2023-12-12T09:52:13.281955+0000","last_fullsized":"2023-12-12T09:52:13.281955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:21:02.014413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050630+0000","last_change":"2023-12-12T09:52:08.020343+0000","last_active":"2023-12-12T09:52:12.050630+0000","last_peered":"2023-12-12T09:52:12.050630+0000","last_clean":"2023-12-12T09:52:12.050630+0000","last_became_active":"2023-12-12T09:52:08.019981+0000","last_became_peered":"2023-12-12T09:52:08.019981+0000","last_unstale":"2023-12-12T09:52:12.050630+0000","last_undegraded":"2023-12-12T09:52:12.050630+0000","last_fullsized":"2023-12-12T09:52:12.050630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104677+0000","last_change":"2023-12-12T09:52:07.011835+0000","last_active":"2023-12-12T09:52:12.104677+0000","last_peered":"2023-12-12T09:52:12.104677+0000","last_clean":"2023-12-12T09:52:12.104677+0000","last_became_active":"2023-12-12T09:52:07.011505+0000","last_became_peered":"2023-12-12T09:52:07.011505+0000","last_unstale":"2023-12-12T09:52:12.104677+0000","last_undegraded":"2023-12-12T09:52:12.104677+0000","last_fullsized":"2023-12-12T09:52:12.104677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285005+0000","last_change":"2023-12-12T09:52:12.080462+0000","last_active":"2023-12-12T09:52:13.285005+0000","last_peered":"2023-12-12T09:52:13.285005+0000","last_clean":"2023-12-12T09:52:13.285005+0000","last_became_active":"2023-12-12T09:52:12.080128+0000","last_became_peered":"2023-12-12T09:52:12.080128+0000","last_unstale":"2023-12-12T09:52:13.285005+0000","last_undegraded":"2023-12-12T09:52:13.285005+0000","last_fullsized":"2023-12-12T09:52:13.285005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054727+0000","last_change":"2023-12-12T09:52:08.028847+0000","last_active":"2023-12-12T09:52:12.054727+0000","last_peered":"2023-12-12T09:52:12.054727+0000","last_clean":"2023-12-12T09:52:12.054727+0000","last_became_active":"2023-12-12T09:52:08.028537+0000","last_became_peered":"2023-12-12T09:52:08.028537+0000","last_unstale":"2023-12-12T09:52:12.054727+0000","last_undegraded":"2023-12-12T09:52:12.054727+0000","last_fullsized":"2023-12-12T09:52:12.054727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068568+0000","last_change":"2023-12-12T09:52:06.992502+0000","last_active":"2023-12-12T09:52:12.068568+0000","last_peered":"2023-12-12T09:52:12.068568+0000","last_clean":"2023-12-12T09:52:12.068568+0000","last_became_active":"2023-12-12T09:52:06.992218+0000","last_became_peered":"2023-12-12T09:52:06.992218+0000","last_unstale":"2023-12-12T09:52:12.068568+0000","last_undegraded":"2023-12-12T09:52:12.068568+0000","last_fullsized":"2023-12-12T09:52:12.068568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284506+0000","last_change":"2023-12-12T09:52:12.083926+0000","last_active":"2023-12-12T09:52:13.284506+0000","last_peered":"2023-12-12T09:52:13.284506+0000","last_clean":"2023-12-12T09:52:13.284506+0000","last_became_active":"2023-12-12T09:52:12.083322+0000","last_became_peered":"2023-12-12T09:52:12.083322+0000","last_unstale":"2023-12-12T09:52:13.284506+0000","last_undegraded":"2023-12-12T09:52:13.284506+0000","last_fullsized":"2023-12-12T09:52:13.284506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:31.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059740+0000","last_change":"2023-12-12T09:52:08.021868+0000","last_active":"2023-12-12T09:52:12.059740+0000","last_peered":"2023-12-12T09:52:12.059740+0000","last_clean":"2023-12-12T09:52:12.059740+0000","last_became_active":"2023-12-12T09:52:08.020477+0000","last_became_peered":"2023-12-12T09:52:08.020477+0000","last_unstale":"2023-12-12T09:52:12.059740+0000","last_undegraded":"2023-12-12T09:52:12.059740+0000","last_fullsized":"2023-12-12T09:52:12.059740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055893+0000","last_change":"2023-12-12T09:52:07.005983+0000","last_active":"2023-12-12T09:52:12.055893+0000","last_peered":"2023-12-12T09:52:12.055893+0000","last_clean":"2023-12-12T09:52:12.055893+0000","last_became_active":"2023-12-12T09:52:07.005723+0000","last_became_peered":"2023-12-12T09:52:07.005723+0000","last_unstale":"2023-12-12T09:52:12.055893+0000","last_undegraded":"2023-12-12T09:52:12.055893+0000","last_fullsized":"2023-12-12T09:52:12.055893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282866+0000","last_change":"2023-12-12T09:52:12.110411+0000","last_active":"2023-12-12T09:52:13.282866+0000","last_peered":"2023-12-12T09:52:13.282866+0000","last_clean":"2023-12-12T09:52:13.282866+0000","last_became_active":"2023-12-12T09:52:12.110045+0000","last_became_peered":"2023-12-12T09:52:12.110045+0000","last_unstale":"2023-12-12T09:52:13.282866+0000","last_undegraded":"2023-12-12T09:52:13.282866+0000","last_fullsized":"2023-12-12T09:52:13.282866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:39.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.069034+0000","last_change":"2023-12-12T09:52:08.034974+0000","last_active":"2023-12-12T09:52:12.069034+0000","last_peered":"2023-12-12T09:52:12.069034+0000","last_clean":"2023-12-12T09:52:12.069034+0000","last_became_active":"2023-12-12T09:52:08.034169+0000","last_became_peered":"2023-12-12T09:52:08.034169+0000","last_unstale":"2023-12-12T09:52:12.069034+0000","last_undegraded":"2023-12-12T09:52:12.069034+0000","last_fullsized":"2023-12-12T09:52:12.069034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:47:41.840110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103698+0000","last_change":"2023-12-12T09:52:07.006046+0000","last_active":"2023-12-12T09:52:12.103698+0000","last_peered":"2023-12-12T09:52:12.103698+0000","last_clean":"2023-12-12T09:52:12.103698+0000","last_became_active":"2023-12-12T09:52:07.005742+0000","last_became_peered":"2023-12-12T09:52:07.005742+0000","last_unstale":"2023-12-12T09:52:12.103698+0000","last_undegraded":"2023-12-12T09:52:12.103698+0000","last_fullsized":"2023-12-12T09:52:12.103698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:31.152852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285141+0000","last_change":"2023-12-12T09:52:12.084173+0000","last_active":"2023-12-12T09:52:13.285141+0000","last_peered":"2023-12-12T09:52:13.285141+0000","last_clean":"2023-12-12T09:52:13.285141+0000","last_became_active":"2023-12-12T09:52:12.083324+0000","last_became_peered":"2023-12-12T09:52:12.083324+0000","last_unstale":"2023-12-12T09:52:13.285141+0000","last_undegraded":"2023-12-12T09:52:13.285141+0000","last_fullsized":"2023-12-12T09:52:13.285141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.071231+0000","last_change":"2023-12-12T09:52:08.022538+0000","last_active":"2023-12-12T09:52:12.071231+0000","last_peered":"2023-12-12T09:52:12.071231+0000","last_clean":"2023-12-12T09:52:12.071231+0000","last_became_active":"2023-12-12T09:52:08.022274+0000","last_became_peered":"2023-12-12T09:52:08.022274+0000","last_unstale":"2023-12-12T09:52:12.071231+0000","last_undegraded":"2023-12-12T09:52:12.071231+0000","last_fullsized":"2023-12-12T09:52:12.071231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053881+0000","last_change":"2023-12-12T09:52:07.012094+0000","last_active":"2023-12-12T09:52:12.053881+0000","last_peered":"2023-12-12T09:52:12.053881+0000","last_clean":"2023-12-12T09:52:12.053881+0000","last_became_active":"2023-12-12T09:52:07.011693+0000","last_became_peered":"2023-12-12T09:52:07.011693+0000","last_unstale":"2023-12-12T09:52:12.053881+0000","last_undegraded":"2023-12-12T09:52:12.053881+0000","last_fullsized":"2023-12-12T09:52:12.053881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:47.890555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281952+0000","last_change":"2023-12-12T09:52:12.111616+0000","last_active":"2023-12-12T09:52:13.281952+0000","last_peered":"2023-12-12T09:52:13.281952+0000","last_clean":"2023-12-12T09:52:13.281952+0000","last_became_active":"2023-12-12T09:52:12.110326+0000","last_became_peered":"2023-12-12T09:52:12.110326+0000","last_unstale":"2023-12-12T09:52:13.281952+0000","last_undegraded":"2023-12-12T09:52:13.281952+0000","last_fullsized":"2023-12-12T09:52:13.281952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:34:01.832456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.063408+0000","last_change":"2023-12-12T09:52:08.034520+0000","last_active":"2023-12-12T09:52:12.063408+0000","last_peered":"2023-12-12T09:52:12.063408+0000","last_clean":"2023-12-12T09:52:12.063408+0000","last_became_active":"2023-12-12T09:52:08.034232+0000","last_became_peered":"2023-12-12T09:52:08.034232+0000","last_unstale":"2023-12-12T09:52:12.063408+0000","last_undegraded":"2023-12-12T09:52:12.063408+0000","last_fullsized":"2023-12-12T09:52:12.063408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.760548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.057080+0000","last_change":"2023-12-12T09:52:07.004065+0000","last_active":"2023-12-12T09:52:12.057080+0000","last_peered":"2023-12-12T09:52:12.057080+0000","last_clean":"2023-12-12T09:52:12.057080+0000","last_became_active":"2023-12-12T09:52:07.003690+0000","last_became_peered":"2023-12-12T09:52:07.003690+0000","last_unstale":"2023-12-12T09:52:12.057080+0000","last_undegraded":"2023-12-12T09:52:12.057080+0000","last_fullsized":"2023-12-12T09:52:12.057080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:47.890555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285159+0000","last_change":"2023-12-12T09:52:12.108409+0000","last_active":"2023-12-12T09:52:13.285159+0000","last_peered":"2023-12-12T09:52:13.285159+0000","last_clean":"2023-12-12T09:52:13.285159+0000","last_became_active":"2023-12-12T09:52:12.107934+0000","last_became_peered":"2023-12-12T09:52:12.107934+0000","last_unstale":"2023-12-12T09:52:13.285159+0000","last_undegraded":"2023-12-12T09:52:13.285159+0000","last_fullsized":"2023-12-12T09:52:13.285159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:20.552141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065611+0000","last_change":"2023-12-12T09:52:08.031660+0000","last_active":"2023-12-12T09:52:12.065611+0000","last_peered":"2023-12-12T09:52:12.065611+0000","last_clean":"2023-12-12T09:52:12.065611+0000","last_became_active":"2023-12-12T09:52:08.030341+0000","last_became_peered":"2023-12-12T09:52:08.030341+0000","last_unstale":"2023-12-12T09:52:12.065611+0000","last_undegraded":"2023-12-12T09:52:12.065611+0000","last_fullsized":"2023-12-12T09:52:12.065611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104424+0000","last_change":"2023-12-12T09:52:07.014640+0000","last_active":"2023-12-12T09:52:12.104424+0000","last_peered":"2023-12-12T09:52:12.104424+0000","last_clean":"2023-12-12T09:52:12.104424+0000","last_became_active":"2023-12-12T09:52:07.014313+0000","last_became_peered":"2023-12-12T09:52:07.014313+0000","last_unstale":"2023-12-12T09:52:12.104424+0000","last_undegraded":"2023-12-12T09:52:12.104424+0000","last_fullsized":"2023-12-12T09:52:12.104424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:25.514763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285307+0000","last_change":"2023-12-12T09:52:12.111140+0000","last_active":"2023-12-12T09:52:13.285307+0000","last_peered":"2023-12-12T09:52:13.285307+0000","last_clean":"2023-12-12T09:52:13.285307+0000","last_became_active":"2023-12-12T09:52:12.110437+0000","last_became_peered":"2023-12-12T09:52:12.110437+0000","last_unstale":"2023-12-12T09:52:13.285307+0000","last_undegraded":"2023-12-12T09:52:13.285307+0000","last_fullsized":"2023-12-12T09:52:13.285307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075331+0000","last_change":"2023-12-12T09:52:08.031638+0000","last_active":"2023-12-12T09:52:12.075331+0000","last_peered":"2023-12-12T09:52:12.075331+0000","last_clean":"2023-12-12T09:52:12.075331+0000","last_became_active":"2023-12-12T09:52:08.031330+0000","last_became_peered":"2023-12-12T09:52:08.031330+0000","last_unstale":"2023-12-12T09:52:12.075331+0000","last_undegraded":"2023-12-12T09:52:12.075331+0000","last_fullsized":"2023-12-12T09:52:12.075331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:27.371886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065722+0000","last_change":"2023-12-12T09:52:06.996775+0000","last_active":"2023-12-12T09:52:12.065722+0000","last_peered":"2023-12-12T09:52:12.065722+0000","last_clean":"2023-12-12T09:52:12.065722+0000","last_became_active":"2023-12-12T09:52:06.995618+0000","last_became_peered":"2023-12-12T09:52:06.995618+0000","last_unstale":"2023-12-12T09:52:12.065722+0000","last_undegraded":"2023-12-12T09:52:12.065722+0000","last_fullsized":"2023-12-12T09:52:12.065722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:26:34.669025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282930+0000","last_change":"2023-12-12T09:52:12.085685+0000","last_active":"2023-12-12T09:52:13.282930+0000","last_peered":"2023-12-12T09:52:13.282930+0000","last_clean":"2023-12-12T09:52:13.282930+0000","last_became_active":"2023-12-12T09:52:12.085247+0000","last_became_peered":"2023-12-12T09:52:12.085247+0000","last_unstale":"2023-12-12T09:52:13.282930+0000","last_undegraded":"2023-12-12T09:52:13.282930+0000","last_fullsized":"2023-12-12T09:52:13.282930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:31.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070066+0000","last_change":"2023-12-12T09:52:08.020254+0000","last_active":"2023-12-12T09:52:12.070066+0000","last_peered":"2023-12-12T09:52:12.070066+0000","last_clean":"2023-12-12T09:52:12.070066+0000","last_became_active":"2023-12-12T09:52:08.019979+0000","last_became_peered":"2023-12-12T09:52:08.019979+0000","last_unstale":"2023-12-12T09:52:12.070066+0000","last_undegraded":"2023-12-12T09:52:12.070066+0000","last_fullsized":"2023-12-12T09:52:12.070066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:51:32.163069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074498+0000","last_change":"2023-12-12T09:52:06.997560+0000","last_active":"2023-12-12T09:52:12.074498+0000","last_peered":"2023-12-12T09:52:12.074498+0000","last_clean":"2023-12-12T09:52:12.074498+0000","last_became_active":"2023-12-12T09:52:06.996940+0000","last_became_peered":"2023-12-12T09:52:06.996940+0000","last_unstale":"2023-12-12T09:52:12.074498+0000","last_undegraded":"2023-12-12T09:52:12.074498+0000","last_fullsized":"2023-12-12T09:52:12.074498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:09.895493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285696+0000","last_change":"2023-12-12T09:52:12.077447+0000","last_active":"2023-12-12T09:52:13.285696+0000","last_peered":"2023-12-12T09:52:13.285696+0000","last_clean":"2023-12-12T09:52:13.285696+0000","last_became_active":"2023-12-12T09:52:12.076927+0000","last_became_peered":"2023-12-12T09:52:12.076927+0000","last_unstale":"2023-12-12T09:52:13.285696+0000","last_undegraded":"2023-12-12T09:52:13.285696+0000","last_fullsized":"2023-12-12T09:52:13.285696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:55.497789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.057627+0000","last_change":"2023-12-12T09:52:08.035111+0000","last_active":"2023-12-12T09:52:12.057627+0000","last_peered":"2023-12-12T09:52:12.057627+0000","last_clean":"2023-12-12T09:52:12.057627+0000","last_became_active":"2023-12-12T09:52:08.034823+0000","last_became_peered":"2023-12-12T09:52:08.034823+0000","last_unstale":"2023-12-12T09:52:12.057627+0000","last_undegraded":"2023-12-12T09:52:12.057627+0000","last_fullsized":"2023-12-12T09:52:12.057627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055046+0000","last_change":"2023-12-12T09:52:06.993403+0000","last_active":"2023-12-12T09:52:12.055046+0000","last_peered":"2023-12-12T09:52:12.055046+0000","last_clean":"2023-12-12T09:52:12.055046+0000","last_became_active":"2023-12-12T09:52:06.993116+0000","last_became_peered":"2023-12-12T09:52:06.993116+0000","last_unstale":"2023-12-12T09:52:12.055046+0000","last_undegraded":"2023-12-12T09:52:12.055046+0000","last_fullsized":"2023-12-12T09:52:12.055046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283254+0000","last_change":"2023-12-12T09:52:12.080184+0000","last_active":"2023-12-12T09:52:13.283254+0000","last_peered":"2023-12-12T09:52:13.283254+0000","last_clean":"2023-12-12T09:52:13.283254+0000","last_became_active":"2023-12-12T09:52:12.079804+0000","last_became_peered":"2023-12-12T09:52:12.079804+0000","last_unstale":"2023-12-12T09:52:13.283254+0000","last_undegraded":"2023-12-12T09:52:13.283254+0000","last_fullsized":"2023-12-12T09:52:13.283254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:27.041979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065376+0000","last_change":"2023-12-12T09:52:08.026908+0000","last_active":"2023-12-12T09:52:12.065376+0000","last_peered":"2023-12-12T09:52:12.065376+0000","last_clean":"2023-12-12T09:52:12.065376+0000","last_became_active":"2023-12-12T09:52:08.025912+0000","last_became_peered":"2023-12-12T09:52:08.025912+0000","last_unstale":"2023-12-12T09:52:12.065376+0000","last_undegraded":"2023-12-12T09:52:12.065376+0000","last_fullsized":"2023-12-12T09:52:12.065376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103688+0000","last_change":"2023-12-12T09:52:07.009290+0000","last_active":"2023-12-12T09:52:12.103688+0000","last_peered":"2023-12-12T09:52:12.103688+0000","last_clean":"2023-12-12T09:52:12.103688+0000","last_became_active":"2023-12-12T09:52:07.008896+0000","last_became_peered":"2023-12-12T09:52:07.008896+0000","last_unstale":"2023-12-12T09:52:12.103688+0000","last_undegraded":"2023-12-12T09:52:12.103688+0000","last_fullsized":"2023-12-12T09:52:12.103688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:21.191123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.290045+0000","last_change":"2023-12-12T09:52:12.115265+0000","last_active":"2023-12-12T09:52:13.290045+0000","last_peered":"2023-12-12T09:52:13.290045+0000","last_clean":"2023-12-12T09:52:13.290045+0000","last_became_active":"2023-12-12T09:52:12.114674+0000","last_became_peered":"2023-12-12T09:52:12.114674+0000","last_unstale":"2023-12-12T09:52:13.290045+0000","last_undegraded":"2023-12-12T09:52:13.290045+0000","last_fullsized":"2023-12-12T09:52:13.290045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:34:54.772153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060809+0000","last_change":"2023-12-12T09:52:08.028299+0000","last_active":"2023-12-12T09:52:12.060809+0000","last_peered":"2023-12-12T09:52:12.060809+0000","last_clean":"2023-12-12T09:52:12.060809+0000","last_became_active":"2023-12-12T09:52:08.028080+0000","last_became_peered":"2023-12-12T09:52:08.028080+0000","last_unstale":"2023-12-12T09:52:12.060809+0000","last_undegraded":"2023-12-12T09:52:12.060809+0000","last_fullsized":"2023-12-12T09:52:12.060809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074571+0000","last_change":"2023-12-12T09:52:06.996346+0000","last_active":"2023-12-12T09:52:12.074571+0000","last_peered":"2023-12-12T09:52:12.074571+0000","last_clean":"2023-12-12T09:52:12.074571+0000","last_became_active":"2023-12-12T09:52:06.995462+0000","last_became_peered":"2023-12-12T09:52:06.995462+0000","last_unstale":"2023-12-12T09:52:12.074571+0000","last_undegraded":"2023-12-12T09:52:12.074571+0000","last_fullsized":"2023-12-12T09:52:12.074571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:59.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103796+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:52:12.103796+0000","last_peered":"2023-12-12T09:52:12.103796+0000","last_clean":"2023-12-12T09:52:12.103796+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:52:12.103796+0000","last_undegraded":"2023-12-12T09:52:12.103796+0000","last_fullsized":"2023-12-12T09:52:12.103796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.055422+0000","last_change":"2023-12-12T09:52:12.109683+0000","last_active":"2023-12-12T09:52:13.055422+0000","last_peered":"2023-12-12T09:52:13.055422+0000","last_clean":"2023-12-12T09:52:13.055422+0000","last_became_active":"2023-12-12T09:52:12.108319+0000","last_became_peered":"2023-12-12T09:52:12.108319+0000","last_unstale":"2023-12-12T09:52:13.055422+0000","last_undegraded":"2023-12-12T09:52:13.055422+0000","last_fullsized":"2023-12-12T09:52:13.055422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:34:01.832456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104240+0000","last_change":"2023-12-12T09:52:08.033828+0000","last_active":"2023-12-12T09:52:12.104240+0000","last_peered":"2023-12-12T09:52:12.104240+0000","last_clean":"2023-12-12T09:52:12.104240+0000","last_became_active":"2023-12-12T09:52:08.030396+0000","last_became_peered":"2023-12-12T09:52:08.030396+0000","last_unstale":"2023-12-12T09:52:12.104240+0000","last_undegraded":"2023-12-12T09:52:12.104240+0000","last_fullsized":"2023-12-12T09:52:12.104240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:11:27.371886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060933+0000","last_change":"2023-12-12T09:52:06.998515+0000","last_active":"2023-12-12T09:52:12.060933+0000","last_peered":"2023-12-12T09:52:12.060933+0000","last_clean":"2023-12-12T09:52:12.060933+0000","last_became_active":"2023-12-12T09:52:06.998220+0000","last_became_peered":"2023-12-12T09:52:06.998220+0000","last_unstale":"2023-12-12T09:52:12.060933+0000","last_undegraded":"2023-12-12T09:52:12.060933+0000","last_fullsized":"2023-12-12T09:52:12.060933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:19.282948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.054359+0000","last_change":"2023-12-12T09:52:12.106813+0000","last_active":"2023-12-12T09:52:13.054359+0000","last_peered":"2023-12-12T09:52:13.054359+0000","last_clean":"2023-12-12T09:52:13.054359+0000","last_became_active":"2023-12-12T09:52:12.106334+0000","last_became_peered":"2023-12-12T09:52:12.106334+0000","last_unstale":"2023-12-12T09:52:13.054359+0000","last_undegraded":"2023-12-12T09:52:13.054359+0000","last_fullsized":"2023-12-12T09:52:13.054359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:39.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074947+0000","last_change":"2023-12-12T09:52:08.033424+0000","last_active":"2023-12-12T09:52:12.074947+0000","last_peered":"2023-12-12T09:52:12.074947+0000","last_clean":"2023-12-12T09:52:12.074947+0000","last_became_active":"2023-12-12T09:52:08.030055+0000","last_became_peered":"2023-12-12T09:52:08.030055+0000","last_unstale":"2023-12-12T09:52:12.074947+0000","last_undegraded":"2023-12-12T09:52:12.074947+0000","last_fullsized":"2023-12-12T09:52:12.074947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:32.450565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056464+0000","last_change":"2023-12-12T09:52:07.000843+0000","last_active":"2023-12-12T09:52:12.056464+0000","last_peered":"2023-12-12T09:52:12.056464+0000","last_clean":"2023-12-12T09:52:12.056464+0000","last_became_active":"2023-12-12T09:52:07.000476+0000","last_became_peered":"2023-12-12T09:52:07.000476+0000","last_unstale":"2023-12-12T09:52:12.056464+0000","last_undegraded":"2023-12-12T09:52:12.056464+0000","last_fullsized":"2023-12-12T09:52:12.056464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:59.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285509+0000","last_change":"2023-12-12T09:52:12.082760+0000","last_active":"2023-12-12T09:52:13.285509+0000","last_peered":"2023-12-12T09:52:13.285509+0000","last_clean":"2023-12-12T09:52:13.285509+0000","last_became_active":"2023-12-12T09:52:12.082269+0000","last_became_peered":"2023-12-12T09:52:12.082269+0000","last_unstale":"2023-12-12T09:52:13.285509+0000","last_undegraded":"2023-12-12T09:52:13.285509+0000","last_fullsized":"2023-12-12T09:52:13.285509+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104860+0000","last_change":"2023-12-12T09:52:08.035636+0000","last_active":"2023-12-12T09:52:12.104860+0000","last_peered":"2023-12-12T09:52:12.104860+0000","last_clean":"2023-12-12T09:52:12.104860+0000","last_became_active":"2023-12-12T09:52:08.035306+0000","last_became_peered":"2023-12-12T09:52:08.035306+0000","last_unstale":"2023-12-12T09:52:12.104860+0000","last_undegraded":"2023-12-12T09:52:12.104860+0000","last_fullsized":"2023-12-12T09:52:12.104860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:34:50.708538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053859+0000","last_change":"2023-12-12T09:52:06.993300+0000","last_active":"2023-12-12T09:52:12.053859+0000","last_peered":"2023-12-12T09:52:12.053859+0000","last_clean":"2023-12-12T09:52:12.053859+0000","last_became_active":"2023-12-12T09:52:06.992970+0000","last_became_peered":"2023-12-12T09:52:06.992970+0000","last_unstale":"2023-12-12T09:52:12.053859+0000","last_undegraded":"2023-12-12T09:52:12.053859+0000","last_fullsized":"2023-12-12T09:52:12.053859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:21.191123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.057884+0000","last_change":"2023-12-12T09:52:12.107302+0000","last_active":"2023-12-12T09:52:13.057884+0000","last_peered":"2023-12-12T09:52:13.057884+0000","last_clean":"2023-12-12T09:52:13.057884+0000","last_became_active":"2023-12-12T09:52:12.106810+0000","last_became_peered":"2023-12-12T09:52:12.106810+0000","last_unstale":"2023-12-12T09:52:13.057884+0000","last_undegraded":"2023-12-12T09:52:13.057884+0000","last_fullsized":"2023-12-12T09:52:13.057884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:25:03.440488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104689+0000","last_change":"2023-12-12T09:52:08.039951+0000","last_active":"2023-12-12T09:52:12.104689+0000","last_peered":"2023-12-12T09:52:12.104689+0000","last_clean":"2023-12-12T09:52:12.104689+0000","last_became_active":"2023-12-12T09:52:08.039124+0000","last_became_peered":"2023-12-12T09:52:08.039124+0000","last_unstale":"2023-12-12T09:52:12.104689+0000","last_undegraded":"2023-12-12T09:52:12.104689+0000","last_fullsized":"2023-12-12T09:52:12.104689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:32.450565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066373+0000","last_change":"2023-12-12T09:52:06.998901+0000","last_active":"2023-12-12T09:52:12.066373+0000","last_peered":"2023-12-12T09:52:12.066373+0000","last_clean":"2023-12-12T09:52:12.066373+0000","last_became_active":"2023-12-12T09:52:06.998458+0000","last_became_peered":"2023-12-12T09:52:06.998458+0000","last_unstale":"2023-12-12T09:52:12.066373+0000","last_undegraded":"2023-12-12T09:52:12.066373+0000","last_fullsized":"2023-12-12T09:52:12.066373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:22:13.759824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.281456+0000","last_change":"2023-12-12T09:52:12.080392+0000","last_active":"2023-12-12T09:52:13.281456+0000","last_peered":"2023-12-12T09:52:13.281456+0000","last_clean":"2023-12-12T09:52:13.281456+0000","last_became_active":"2023-12-12T09:52:12.079941+0000","last_became_peered":"2023-12-12T09:52:12.079941+0000","last_unstale":"2023-12-12T09:52:13.281456+0000","last_undegraded":"2023-12-12T09:52:13.281456+0000","last_fullsized":"2023-12-12T09:52:13.281456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:10.824163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103824+0000","last_change":"2023-12-12T09:52:08.032605+0000","last_active":"2023-12-12T09:52:12.103824+0000","last_peered":"2023-12-12T09:52:12.103824+0000","last_clean":"2023-12-12T09:52:12.103824+0000","last_became_active":"2023-12-12T09:52:08.032321+0000","last_became_peered":"2023-12-12T09:52:08.032321+0000","last_unstale":"2023-12-12T09:52:12.103824+0000","last_undegraded":"2023-12-12T09:52:12.103824+0000","last_fullsized":"2023-12-12T09:52:12.103824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053766+0000","last_change":"2023-12-12T09:52:06.993710+0000","last_active":"2023-12-12T09:52:12.053766+0000","last_peered":"2023-12-12T09:52:12.053766+0000","last_clean":"2023-12-12T09:52:12.053766+0000","last_became_active":"2023-12-12T09:52:06.993427+0000","last_became_peered":"2023-12-12T09:52:06.993427+0000","last_unstale":"2023-12-12T09:52:12.053766+0000","last_undegraded":"2023-12-12T09:52:12.053766+0000","last_fullsized":"2023-12-12T09:52:12.053766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:39.005895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.289994+0000","last_change":"2023-12-12T09:52:12.083094+0000","last_active":"2023-12-12T09:52:13.289994+0000","last_peered":"2023-12-12T09:52:13.289994+0000","last_clean":"2023-12-12T09:52:13.289994+0000","last_became_active":"2023-12-12T09:52:12.082050+0000","last_became_peered":"2023-12-12T09:52:12.082050+0000","last_unstale":"2023-12-12T09:52:13.289994+0000","last_undegraded":"2023-12-12T09:52:13.289994+0000","last_fullsized":"2023-12-12T09:52:13.289994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:51.971159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050599+0000","last_change":"2023-12-12T09:52:08.020100+0000","last_active":"2023-12-12T09:52:12.050599+0000","last_peered":"2023-12-12T09:52:12.050599+0000","last_clean":"2023-12-12T09:52:12.050599+0000","last_became_active":"2023-12-12T09:52:08.019776+0000","last_became_peered":"2023-12-12T09:52:08.019776+0000","last_unstale":"2023-12-12T09:52:12.050599+0000","last_undegraded":"2023-12-12T09:52:12.050599+0000","last_fullsized":"2023-12-12T09:52:12.050599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050514+0000","last_change":"2023-12-12T09:52:07.002330+0000","last_active":"2023-12-12T09:52:12.050514+0000","last_peered":"2023-12-12T09:52:12.050514+0000","last_clean":"2023-12-12T09:52:12.050514+0000","last_became_active":"2023-12-12T09:52:07.001838+0000","last_became_peered":"2023-12-12T09:52:07.001838+0000","last_unstale":"2023-12-12T09:52:12.050514+0000","last_undegraded":"2023-12-12T09:52:12.050514+0000","last_fullsized":"2023-12-12T09:52:12.050514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:44.065611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.287475+0000","last_change":"2023-12-12T09:52:12.111595+0000","last_active":"2023-12-12T09:52:13.287475+0000","last_peered":"2023-12-12T09:52:13.287475+0000","last_clean":"2023-12-12T09:52:13.287475+0000","last_became_active":"2023-12-12T09:52:12.111143+0000","last_became_peered":"2023-12-12T09:52:12.111143+0000","last_unstale":"2023-12-12T09:52:13.287475+0000","last_undegraded":"2023-12-12T09:52:13.287475+0000","last_fullsized":"2023-12-12T09:52:13.287475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:07:22.383529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104101+0000","last_change":"2023-12-12T09:52:08.036655+0000","last_active":"2023-12-12T09:52:12.104101+0000","last_peered":"2023-12-12T09:52:12.104101+0000","last_clean":"2023-12-12T09:52:12.104101+0000","last_became_active":"2023-12-12T09:52:08.036260+0000","last_became_peered":"2023-12-12T09:52:08.036260+0000","last_unstale":"2023-12-12T09:52:12.104101+0000","last_undegraded":"2023-12-12T09:52:12.104101+0000","last_fullsized":"2023-12-12T09:52:12.104101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:35.813548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074437+0000","last_change":"2023-12-12T09:52:06.999233+0000","last_active":"2023-12-12T09:52:12.074437+0000","last_peered":"2023-12-12T09:52:12.074437+0000","last_clean":"2023-12-12T09:52:12.074437+0000","last_became_active":"2023-12-12T09:52:06.998962+0000","last_became_peered":"2023-12-12T09:52:06.998962+0000","last_unstale":"2023-12-12T09:52:12.074437+0000","last_undegraded":"2023-12-12T09:52:12.074437+0000","last_fullsized":"2023-12-12T09:52:12.074437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:25.514763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.285663+0000","last_change":"2023-12-12T09:52:12.106941+0000","last_active":"2023-12-12T09:52:13.285663+0000","last_peered":"2023-12-12T09:52:13.285663+0000","last_clean":"2023-12-12T09:52:13.285663+0000","last_became_active":"2023-12-12T09:52:12.106547+0000","last_became_peered":"2023-12-12T09:52:12.106547+0000","last_unstale":"2023-12-12T09:52:13.285663+0000","last_undegraded":"2023-12-12T09:52:13.285663+0000","last_fullsized":"2023-12-12T09:52:13.285663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:04:27.791759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070910+0000","last_change":"2023-12-12T09:52:08.029830+0000","last_active":"2023-12-12T09:52:12.070910+0000","last_peered":"2023-12-12T09:52:12.070910+0000","last_clean":"2023-12-12T09:52:12.070910+0000","last_became_active":"2023-12-12T09:52:08.029455+0000","last_became_peered":"2023-12-12T09:52:08.029455+0000","last_unstale":"2023-12-12T09:52:12.070910+0000","last_undegraded":"2023-12-12T09:52:12.070910+0000","last_fullsized":"2023-12-12T09:52:12.070910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068345+0000","last_change":"2023-12-12T09:52:07.000933+0000","last_active":"2023-12-12T09:52:12.068345+0000","last_peered":"2023-12-12T09:52:12.068345+0000","last_clean":"2023-12-12T09:52:12.068345+0000","last_became_active":"2023-12-12T09:52:07.000646+0000","last_became_peered":"2023-12-12T09:52:07.000646+0000","last_unstale":"2023-12-12T09:52:12.068345+0000","last_undegraded":"2023-12-12T09:52:12.068345+0000","last_fullsized":"2023-12-12T09:52:12.068345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:44.065611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.059459+0000","last_change":"2023-12-12T09:52:12.110657+0000","last_active":"2023-12-12T09:52:13.059459+0000","last_peered":"2023-12-12T09:52:13.059459+0000","last_clean":"2023-12-12T09:52:13.059459+0000","last_became_active":"2023-12-12T09:52:12.109951+0000","last_became_peered":"2023-12-12T09:52:12.109951+0000","last_unstale":"2023-12-12T09:52:13.059459+0000","last_undegraded":"2023-12-12T09:52:13.059459+0000","last_fullsized":"2023-12-12T09:52:13.059459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:20.552141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104135+0000","last_change":"2023-12-12T09:52:08.035067+0000","last_active":"2023-12-12T09:52:12.104135+0000","last_peered":"2023-12-12T09:52:12.104135+0000","last_clean":"2023-12-12T09:52:12.104135+0000","last_became_active":"2023-12-12T09:52:08.034803+0000","last_became_peered":"2023-12-12T09:52:08.034803+0000","last_unstale":"2023-12-12T09:52:12.104135+0000","last_undegraded":"2023-12-12T09:52:12.104135+0000","last_fullsized":"2023-12-12T09:52:12.104135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:22.978364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:27.255184+0000","last_change":"2023-12-12T09:52:06.998675+0000","last_active":"2023-12-12T09:52:27.255184+0000","last_peered":"2023-12-12T09:52:27.255184+0000","last_clean":"2023-12-12T09:52:27.255184+0000","last_became_active":"2023-12-12T09:52:06.998364+0000","last_became_peered":"2023-12-12T09:52:06.998364+0000","last_unstale":"2023-12-12T09:52:27.255184+0000","last_undegraded":"2023-12-12T09:52:27.255184+0000","last_fullsized":"2023-12-12T09:52:27.255184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3839,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283450+0000","last_change":"2023-12-12T09:52:12.086437+0000","last_active":"2023-12-12T09:52:13.283450+0000","last_peered":"2023-12-12T09:52:13.283450+0000","last_clean":"2023-12-12T09:52:13.283450+0000","last_became_active":"2023-12-12T09:52:12.083068+0000","last_became_peered":"2023-12-12T09:52:12.083068+0000","last_unstale":"2023-12-12T09:52:13.283450+0000","last_undegraded":"2023-12-12T09:52:13.283450+0000","last_fullsized":"2023-12-12T09:52:13.283450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:36.514180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066756+0000","last_change":"2023-12-12T09:52:08.030198+0000","last_active":"2023-12-12T09:52:12.066756+0000","last_peered":"2023-12-12T09:52:12.066756+0000","last_clean":"2023-12-12T09:52:12.066756+0000","last_became_active":"2023-12-12T09:52:08.027733+0000","last_became_peered":"2023-12-12T09:52:08.027733+0000","last_unstale":"2023-12-12T09:52:12.066756+0000","last_undegraded":"2023-12-12T09:52:12.066756+0000","last_fullsized":"2023-12-12T09:52:12.066756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:08.788149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056359+0000","last_change":"2023-12-12T09:52:07.004177+0000","last_active":"2023-12-12T09:52:12.056359+0000","last_peered":"2023-12-12T09:52:12.056359+0000","last_clean":"2023-12-12T09:52:12.056359+0000","last_became_active":"2023-12-12T09:52:07.003836+0000","last_became_peered":"2023-12-12T09:52:07.003836+0000","last_unstale":"2023-12-12T09:52:12.056359+0000","last_undegraded":"2023-12-12T09:52:12.056359+0000","last_fullsized":"2023-12-12T09:52:12.056359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284483+0000","last_change":"2023-12-12T09:52:12.081391+0000","last_active":"2023-12-12T09:52:13.284483+0000","last_peered":"2023-12-12T09:52:13.284483+0000","last_clean":"2023-12-12T09:52:13.284483+0000","last_became_active":"2023-12-12T09:52:12.080949+0000","last_became_peered":"2023-12-12T09:52:12.080949+0000","last_unstale":"2023-12-12T09:52:13.284483+0000","last_undegraded":"2023-12-12T09:52:13.284483+0000","last_fullsized":"2023-12-12T09:52:13.284483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:46:39.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070164+0000","last_change":"2023-12-12T09:52:08.023308+0000","last_active":"2023-12-12T09:52:12.070164+0000","last_peered":"2023-12-12T09:52:12.070164+0000","last_clean":"2023-12-12T09:52:12.070164+0000","last_became_active":"2023-12-12T09:52:08.023048+0000","last_became_peered":"2023-12-12T09:52:08.023048+0000","last_unstale":"2023-12-12T09:52:12.070164+0000","last_undegraded":"2023-12-12T09:52:12.070164+0000","last_fullsized":"2023-12-12T09:52:12.070164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075599+0000","last_change":"2023-12-12T09:52:07.001746+0000","last_active":"2023-12-12T09:52:12.075599+0000","last_peered":"2023-12-12T09:52:12.075599+0000","last_clean":"2023-12-12T09:52:12.075599+0000","last_became_active":"2023-12-12T09:52:07.001424+0000","last_became_peered":"2023-12-12T09:52:07.001424+0000","last_unstale":"2023-12-12T09:52:12.075599+0000","last_undegraded":"2023-12-12T09:52:12.075599+0000","last_fullsized":"2023-12-12T09:52:12.075599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:54.297647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.282877+0000","last_change":"2023-12-12T09:52:12.077880+0000","last_active":"2023-12-12T09:52:13.282877+0000","last_peered":"2023-12-12T09:52:13.282877+0000","last_clean":"2023-12-12T09:52:13.282877+0000","last_became_active":"2023-12-12T09:52:12.077365+0000","last_became_peered":"2023-12-12T09:52:12.077365+0000","last_unstale":"2023-12-12T09:52:13.282877+0000","last_undegraded":"2023-12-12T09:52:13.282877+0000","last_fullsized":"2023-12-12T09:52:13.282877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:43:05.307563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075525+0000","last_change":"2023-12-12T09:52:08.029359+0000","last_active":"2023-12-12T09:52:12.075525+0000","last_peered":"2023-12-12T09:52:12.075525+0000","last_clean":"2023-12-12T09:52:12.075525+0000","last_became_active":"2023-12-12T09:52:08.028997+0000","last_became_peered":"2023-12-12T09:52:08.028997+0000","last_unstale":"2023-12-12T09:52:12.075525+0000","last_undegraded":"2023-12-12T09:52:12.075525+0000","last_fullsized":"2023-12-12T09:52:12.075525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:22.978364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.052142+0000","last_change":"2023-12-12T09:52:07.006953+0000","last_active":"2023-12-12T09:52:12.052142+0000","last_peered":"2023-12-12T09:52:12.052142+0000","last_clean":"2023-12-12T09:52:12.052142+0000","last_became_active":"2023-12-12T09:52:07.006666+0000","last_became_peered":"2023-12-12T09:52:07.006666+0000","last_unstale":"2023-12-12T09:52:12.052142+0000","last_undegraded":"2023-12-12T09:52:12.052142+0000","last_fullsized":"2023-12-12T09:52:12.052142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:26:34.669025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.283756+0000","last_change":"2023-12-12T09:52:12.104433+0000","last_active":"2023-12-12T09:52:13.283756+0000","last_peered":"2023-12-12T09:52:13.283756+0000","last_clean":"2023-12-12T09:52:13.283756+0000","last_became_active":"2023-12-12T09:52:12.104126+0000","last_became_peered":"2023-12-12T09:52:12.104126+0000","last_unstale":"2023-12-12T09:52:13.283756+0000","last_undegraded":"2023-12-12T09:52:13.283756+0000","last_fullsized":"2023-12-12T09:52:13.283756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:25:03.440488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.289874+0000","last_change":"2023-12-12T09:52:12.111376+0000","last_active":"2023-12-12T09:52:13.289874+0000","last_peered":"2023-12-12T09:52:13.289874+0000","last_clean":"2023-12-12T09:52:13.289874+0000","last_became_active":"2023-12-12T09:52:12.110863+0000","last_became_peered":"2023-12-12T09:52:12.110863+0000","last_unstale":"2023-12-12T09:52:13.289874+0000","last_undegraded":"2023-12-12T09:52:13.289874+0000","last_fullsized":"2023-12-12T09:52:13.289874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:55.497789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055671+0000","last_change":"2023-12-12T09:52:08.024511+0000","last_active":"2023-12-12T09:52:12.055671+0000","last_peered":"2023-12-12T09:52:12.055671+0000","last_clean":"2023-12-12T09:52:12.055671+0000","last_became_active":"2023-12-12T09:52:08.024258+0000","last_became_peered":"2023-12-12T09:52:08.024258+0000","last_unstale":"2023-12-12T09:52:12.055671+0000","last_undegraded":"2023-12-12T09:52:12.055671+0000","last_fullsized":"2023-12-12T09:52:12.055671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074250+0000","last_change":"2023-12-12T09:52:07.000641+0000","last_active":"2023-12-12T09:52:12.074250+0000","last_peered":"2023-12-12T09:52:12.074250+0000","last_clean":"2023-12-12T09:52:12.074250+0000","last_became_active":"2023-12-12T09:52:07.000245+0000","last_became_peered":"2023-12-12T09:52:07.000245+0000","last_unstale":"2023-12-12T09:52:12.074250+0000","last_undegraded":"2023-12-12T09:52:12.074250+0000","last_fullsized":"2023-12-12T09:52:12.074250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:46.546348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.056067+0000","last_change":"2023-12-12T09:52:12.078229+0000","last_active":"2023-12-12T09:52:13.056067+0000","last_peered":"2023-12-12T09:52:13.056067+0000","last_clean":"2023-12-12T09:52:13.056067+0000","last_became_active":"2023-12-12T09:52:12.077764+0000","last_became_peered":"2023-12-12T09:52:12.077764+0000","last_unstale":"2023-12-12T09:52:13.056067+0000","last_undegraded":"2023-12-12T09:52:13.056067+0000","last_fullsized":"2023-12-12T09:52:13.056067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:21:02.014413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059639+0000","last_change":"2023-12-12T09:52:08.028827+0000","last_active":"2023-12-12T09:52:12.059639+0000","last_peered":"2023-12-12T09:52:12.059639+0000","last_clean":"2023-12-12T09:52:12.059639+0000","last_became_active":"2023-12-12T09:52:08.024860+0000","last_became_peered":"2023-12-12T09:52:08.024860+0000","last_unstale":"2023-12-12T09:52:12.059639+0000","last_undegraded":"2023-12-12T09:52:12.059639+0000","last_fullsized":"2023-12-12T09:52:12.059639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054975+0000","last_change":"2023-12-12T09:52:06.999381+0000","last_active":"2023-12-12T09:52:12.054975+0000","last_peered":"2023-12-12T09:52:12.054975+0000","last_clean":"2023-12-12T09:52:12.054975+0000","last_became_active":"2023-12-12T09:52:06.999099+0000","last_became_peered":"2023-12-12T09:52:06.999099+0000","last_unstale":"2023-12-12T09:52:12.054975+0000","last_undegraded":"2023-12-12T09:52:12.054975+0000","last_fullsized":"2023-12-12T09:52:12.054975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:00.072585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050899+0000","last_change":"2023-12-12T09:52:07.003879+0000","last_active":"2023-12-12T09:52:12.050899+0000","last_peered":"2023-12-12T09:52:12.050899+0000","last_clean":"2023-12-12T09:52:12.050899+0000","last_became_active":"2023-12-12T09:52:07.002760+0000","last_became_peered":"2023-12-12T09:52:07.002760+0000","last_unstale":"2023-12-12T09:52:12.050899+0000","last_undegraded":"2023-12-12T09:52:12.050899+0000","last_fullsized":"2023-12-12T09:52:12.050899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:07.777932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068247+0000","last_change":"2023-12-12T09:52:08.026740+0000","last_active":"2023-12-12T09:52:12.068247+0000","last_peered":"2023-12-12T09:52:12.068247+0000","last_clean":"2023-12-12T09:52:12.068247+0000","last_became_active":"2023-12-12T09:52:08.024698+0000","last_became_peered":"2023-12-12T09:52:08.024698+0000","last_unstale":"2023-12-12T09:52:12.068247+0000","last_undegraded":"2023-12-12T09:52:12.068247+0000","last_fullsized":"2023-12-12T09:52:12.068247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.284952+0000","last_change":"2023-12-12T09:52:12.078748+0000","last_active":"2023-12-12T09:52:13.284952+0000","last_peered":"2023-12-12T09:52:13.284952+0000","last_clean":"2023-12-12T09:52:13.284952+0000","last_became_active":"2023-12-12T09:52:12.077889+0000","last_became_peered":"2023-12-12T09:52:12.077889+0000","last_unstale":"2023-12-12T09:52:13.284952+0000","last_undegraded":"2023-12-12T09:52:13.284952+0000","last_fullsized":"2023-12-12T09:52:13.284952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:05.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054823+0000","last_change":"2023-12-12T09:52:07.008787+0000","last_active":"2023-12-12T09:52:12.054823+0000","last_peered":"2023-12-12T09:52:12.054823+0000","last_clean":"2023-12-12T09:52:12.054823+0000","last_became_active":"2023-12-12T09:52:07.008455+0000","last_became_peered":"2023-12-12T09:52:07.008455+0000","last_unstale":"2023-12-12T09:52:12.054823+0000","last_undegraded":"2023-12-12T09:52:12.054823+0000","last_fullsized":"2023-12-12T09:52:12.054823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:05.946524+0000","last_clean_scrub_stamp":"2023-12-12T09:52:05.946524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:58:33.624982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.061074+0000","last_change":"2023-12-12T09:52:08.022255+0000","last_active":"2023-12-12T09:52:12.061074+0000","last_peered":"2023-12-12T09:52:12.061074+0000","last_clean":"2023-12-12T09:52:12.061074+0000","last_became_active":"2023-12-12T09:52:08.020240+0000","last_became_peered":"2023-12-12T09:52:08.020240+0000","last_unstale":"2023-12-12T09:52:12.061074+0000","last_undegraded":"2023-12-12T09:52:12.061074+0000","last_fullsized":"2023-12-12T09:52:12.061074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:22.201340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-12T09:52:13.286721+0000","last_change":"2023-12-12T09:52:12.081878+0000","last_active":"2023-12-12T09:52:13.286721+0000","last_peered":"2023-12-12T09:52:13.286721+0000","last_clean":"2023-12-12T09:52:13.286721+0000","last_became_active":"2023-12-12T09:52:12.081502+0000","last_became_peered":"2023-12-12T09:52:12.081502+0000","last_unstale":"2023-12-12T09:52:13.286721+0000","last_undegraded":"2023-12-12T09:52:13.286721+0000","last_fullsized":"2023-12-12T09:52:13.286721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:11.020356+0000","last_clean_scrub_stamp":"2023-12-12T09:52:11.020356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:18:41.559574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050317+0000","last_change":"2023-12-12T09:52:08.021237+0000","last_active":"2023-12-12T09:52:12.050317+0000","last_peered":"2023-12-12T09:52:12.050317+0000","last_clean":"2023-12-12T09:52:12.050317+0000","last_became_active":"2023-12-12T09:52:08.019519+0000","last_became_peered":"2023-12-12T09:52:08.019519+0000","last_unstale":"2023-12-12T09:52:12.050317+0000","last_undegraded":"2023-12-12T09:52:12.050317+0000","last_fullsized":"2023-12-12T09:52:12.050317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:01.082803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054863+0000","last_change":"2023-12-12T09:52:08.019363+0000","last_active":"2023-12-12T09:52:12.054863+0000","last_peered":"2023-12-12T09:52:12.054863+0000","last_clean":"2023-12-12T09:52:12.054863+0000","last_became_active":"2023-12-12T09:52:08.012574+0000","last_became_peered":"2023-12-12T09:52:08.012574+0000","last_unstale":"2023-12-12T09:52:12.054863+0000","last_undegraded":"2023-12-12T09:52:12.054863+0000","last_fullsized":"2023-12-12T09:52:12.054863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:08.788149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074382+0000","last_change":"2023-12-12T09:52:08.030165+0000","last_active":"2023-12-12T09:52:12.074382+0000","last_peered":"2023-12-12T09:52:12.074382+0000","last_clean":"2023-12-12T09:52:12.074382+0000","last_became_active":"2023-12-12T09:52:08.029626+0000","last_became_peered":"2023-12-12T09:52:08.029626+0000","last_unstale":"2023-12-12T09:52:12.074382+0000","last_undegraded":"2023-12-12T09:52:12.074382+0000","last_fullsized":"2023-12-12T09:52:12.074382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.068155+0000","last_change":"2023-12-12T09:52:08.040599+0000","last_active":"2023-12-12T09:52:12.068155+0000","last_peered":"2023-12-12T09:52:12.068155+0000","last_clean":"2023-12-12T09:52:12.068155+0000","last_became_active":"2023-12-12T09:52:08.039875+0000","last_became_peered":"2023-12-12T09:52:08.039875+0000","last_unstale":"2023-12-12T09:52:12.068155+0000","last_undegraded":"2023-12-12T09:52:12.068155+0000","last_fullsized":"2023-12-12T09:52:12.068155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.060010+0000","last_change":"2023-12-12T09:52:08.032398+0000","last_active":"2023-12-12T09:52:12.060010+0000","last_peered":"2023-12-12T09:52:12.060010+0000","last_clean":"2023-12-12T09:52:12.060010+0000","last_became_active":"2023-12-12T09:52:08.031194+0000","last_became_peered":"2023-12-12T09:52:08.031194+0000","last_unstale":"2023-12-12T09:52:12.060010+0000","last_undegraded":"2023-12-12T09:52:12.060010+0000","last_fullsized":"2023-12-12T09:52:12.060010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:47.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056859+0000","last_change":"2023-12-12T09:52:08.023831+0000","last_active":"2023-12-12T09:52:12.056859+0000","last_peered":"2023-12-12T09:52:12.056859+0000","last_clean":"2023-12-12T09:52:12.056859+0000","last_became_active":"2023-12-12T09:52:08.023532+0000","last_became_peered":"2023-12-12T09:52:08.023532+0000","last_unstale":"2023-12-12T09:52:12.056859+0000","last_undegraded":"2023-12-12T09:52:12.056859+0000","last_fullsized":"2023-12-12T09:52:12.056859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054670+0000","last_change":"2023-12-12T09:52:08.028457+0000","last_active":"2023-12-12T09:52:12.054670+0000","last_peered":"2023-12-12T09:52:12.054670+0000","last_clean":"2023-12-12T09:52:12.054670+0000","last_became_active":"2023-12-12T09:52:08.028172+0000","last_became_peered":"2023-12-12T09:52:08.028172+0000","last_unstale":"2023-12-12T09:52:12.054670+0000","last_undegraded":"2023-12-12T09:52:12.054670+0000","last_fullsized":"2023-12-12T09:52:12.054670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.063395+0000","last_change":"2023-12-12T09:52:08.033841+0000","last_active":"2023-12-12T09:52:12.063395+0000","last_peered":"2023-12-12T09:52:12.063395+0000","last_clean":"2023-12-12T09:52:12.063395+0000","last_became_active":"2023-12-12T09:52:08.030880+0000","last_became_peered":"2023-12-12T09:52:08.030880+0000","last_unstale":"2023-12-12T09:52:12.063395+0000","last_undegraded":"2023-12-12T09:52:12.063395+0000","last_fullsized":"2023-12-12T09:52:12.063395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074138+0000","last_change":"2023-12-12T09:52:08.032051+0000","last_active":"2023-12-12T09:52:12.074138+0000","last_peered":"2023-12-12T09:52:12.074138+0000","last_clean":"2023-12-12T09:52:12.074138+0000","last_became_active":"2023-12-12T09:52:08.031785+0000","last_became_peered":"2023-12-12T09:52:08.031785+0000","last_unstale":"2023-12-12T09:52:12.074138+0000","last_undegraded":"2023-12-12T09:52:12.074138+0000","last_fullsized":"2023-12-12T09:52:12.074138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054886+0000","last_change":"2023-12-12T09:52:08.031522+0000","last_active":"2023-12-12T09:52:12.054886+0000","last_peered":"2023-12-12T09:52:12.054886+0000","last_clean":"2023-12-12T09:52:12.054886+0000","last_became_active":"2023-12-12T09:52:08.031262+0000","last_became_peered":"2023-12-12T09:52:08.031262+0000","last_unstale":"2023-12-12T09:52:12.054886+0000","last_undegraded":"2023-12-12T09:52:12.054886+0000","last_fullsized":"2023-12-12T09:52:12.054886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074154+0000","last_change":"2023-12-12T09:52:08.024395+0000","last_active":"2023-12-12T09:52:12.074154+0000","last_peered":"2023-12-12T09:52:12.074154+0000","last_clean":"2023-12-12T09:52:12.074154+0000","last_became_active":"2023-12-12T09:52:08.024105+0000","last_became_peered":"2023-12-12T09:52:08.024105+0000","last_unstale":"2023-12-12T09:52:12.074154+0000","last_undegraded":"2023-12-12T09:52:12.074154+0000","last_fullsized":"2023-12-12T09:52:12.074154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.760548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.069490+0000","last_change":"2023-12-12T09:52:08.037425+0000","last_active":"2023-12-12T09:52:12.069490+0000","last_peered":"2023-12-12T09:52:12.069490+0000","last_clean":"2023-12-12T09:52:12.069490+0000","last_became_active":"2023-12-12T09:52:08.035756+0000","last_became_peered":"2023-12-12T09:52:08.035756+0000","last_unstale":"2023-12-12T09:52:12.069490+0000","last_undegraded":"2023-12-12T09:52:12.069490+0000","last_fullsized":"2023-12-12T09:52:12.069490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:01.082803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.050903+0000","last_change":"2023-12-12T09:52:08.031988+0000","last_active":"2023-12-12T09:52:12.050903+0000","last_peered":"2023-12-12T09:52:12.050903+0000","last_clean":"2023-12-12T09:52:12.050903+0000","last_became_active":"2023-12-12T09:52:08.031525+0000","last_became_peered":"2023-12-12T09:52:08.031525+0000","last_unstale":"2023-12-12T09:52:12.050903+0000","last_undegraded":"2023-12-12T09:52:12.050903+0000","last_fullsized":"2023-12-12T09:52:12.050903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104392+0000","last_change":"2023-12-12T09:52:08.034992+0000","last_active":"2023-12-12T09:52:12.104392+0000","last_peered":"2023-12-12T09:52:12.104392+0000","last_clean":"2023-12-12T09:52:12.104392+0000","last_became_active":"2023-12-12T09:52:08.034424+0000","last_became_peered":"2023-12-12T09:52:08.034424+0000","last_unstale":"2023-12-12T09:52:12.104392+0000","last_undegraded":"2023-12-12T09:52:12.104392+0000","last_fullsized":"2023-12-12T09:52:12.104392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:24.493525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.053756+0000","last_change":"2023-12-12T09:52:08.021603+0000","last_active":"2023-12-12T09:52:12.053756+0000","last_peered":"2023-12-12T09:52:12.053756+0000","last_clean":"2023-12-12T09:52:12.053756+0000","last_became_active":"2023-12-12T09:52:08.021253+0000","last_became_peered":"2023-12-12T09:52:08.021253+0000","last_unstale":"2023-12-12T09:52:12.053756+0000","last_undegraded":"2023-12-12T09:52:12.053756+0000","last_fullsized":"2023-12-12T09:52:12.053756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:45.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.075822+0000","last_change":"2023-12-12T09:52:08.036026+0000","last_active":"2023-12-12T09:52:12.075822+0000","last_peered":"2023-12-12T09:52:12.075822+0000","last_clean":"2023-12-12T09:52:12.075822+0000","last_became_active":"2023-12-12T09:52:08.035390+0000","last_became_peered":"2023-12-12T09:52:08.035390+0000","last_unstale":"2023-12-12T09:52:12.075822+0000","last_undegraded":"2023-12-12T09:52:12.075822+0000","last_fullsized":"2023-12-12T09:52:12.075822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:51.434174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.073460+0000","last_change":"2023-12-12T09:52:08.013438+0000","last_active":"2023-12-12T09:52:12.073460+0000","last_peered":"2023-12-12T09:52:12.073460+0000","last_clean":"2023-12-12T09:52:12.073460+0000","last_became_active":"2023-12-12T09:52:08.013002+0000","last_became_peered":"2023-12-12T09:52:08.013002+0000","last_unstale":"2023-12-12T09:52:12.073460+0000","last_undegraded":"2023-12-12T09:52:12.073460+0000","last_fullsized":"2023-12-12T09:52:12.073460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:28.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055788+0000","last_change":"2023-12-12T09:52:08.032162+0000","last_active":"2023-12-12T09:52:12.055788+0000","last_peered":"2023-12-12T09:52:12.055788+0000","last_clean":"2023-12-12T09:52:12.055788+0000","last_became_active":"2023-12-12T09:52:08.031735+0000","last_became_peered":"2023-12-12T09:52:08.031735+0000","last_unstale":"2023-12-12T09:52:12.055788+0000","last_undegraded":"2023-12-12T09:52:12.055788+0000","last_fullsized":"2023-12-12T09:52:12.055788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.070823+0000","last_change":"2023-12-12T09:52:08.024576+0000","last_active":"2023-12-12T09:52:12.070823+0000","last_peered":"2023-12-12T09:52:12.070823+0000","last_clean":"2023-12-12T09:52:12.070823+0000","last_became_active":"2023-12-12T09:52:08.024277+0000","last_became_peered":"2023-12-12T09:52:08.024277+0000","last_unstale":"2023-12-12T09:52:12.070823+0000","last_undegraded":"2023-12-12T09:52:12.070823+0000","last_fullsized":"2023-12-12T09:52:12.070823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:48.900772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054635+0000","last_change":"2023-12-12T09:52:08.029262+0000","last_active":"2023-12-12T09:52:12.054635+0000","last_peered":"2023-12-12T09:52:12.054635+0000","last_clean":"2023-12-12T09:52:12.054635+0000","last_became_active":"2023-12-12T09:52:08.028949+0000","last_became_peered":"2023-12-12T09:52:08.028949+0000","last_unstale":"2023-12-12T09:52:12.054635+0000","last_undegraded":"2023-12-12T09:52:12.054635+0000","last_fullsized":"2023-12-12T09:52:12.054635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:55.307864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.066542+0000","last_change":"2023-12-12T09:52:08.024637+0000","last_active":"2023-12-12T09:52:12.066542+0000","last_peered":"2023-12-12T09:52:12.066542+0000","last_clean":"2023-12-12T09:52:12.066542+0000","last_became_active":"2023-12-12T09:52:08.023754+0000","last_became_peered":"2023-12-12T09:52:08.023754+0000","last_unstale":"2023-12-12T09:52:12.066542+0000","last_undegraded":"2023-12-12T09:52:12.066542+0000","last_fullsized":"2023-12-12T09:52:12.066542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:45:45.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.055913+0000","last_change":"2023-12-12T09:52:08.024613+0000","last_active":"2023-12-12T09:52:12.055913+0000","last_peered":"2023-12-12T09:52:12.055913+0000","last_clean":"2023-12-12T09:52:12.055913+0000","last_became_active":"2023-12-12T09:52:08.024257+0000","last_became_peered":"2023-12-12T09:52:08.024257+0000","last_unstale":"2023-12-12T09:52:12.055913+0000","last_undegraded":"2023-12-12T09:52:12.055913+0000","last_fullsized":"2023-12-12T09:52:12.055913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:10.905710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.054417+0000","last_change":"2023-12-12T09:52:08.023097+0000","last_active":"2023-12-12T09:52:12.054417+0000","last_peered":"2023-12-12T09:52:12.054417+0000","last_clean":"2023-12-12T09:52:12.054417+0000","last_became_active":"2023-12-12T09:52:08.022866+0000","last_became_peered":"2023-12-12T09:52:08.022866+0000","last_unstale":"2023-12-12T09:52:12.054417+0000","last_undegraded":"2023-12-12T09:52:12.054417+0000","last_fullsized":"2023-12-12T09:52:12.054417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:50:26.524981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104813+0000","last_change":"2023-12-12T09:52:08.039672+0000","last_active":"2023-12-12T09:52:12.104813+0000","last_peered":"2023-12-12T09:52:12.104813+0000","last_clean":"2023-12-12T09:52:12.104813+0000","last_became_active":"2023-12-12T09:52:08.039145+0000","last_became_peered":"2023-12-12T09:52:08.039145+0000","last_unstale":"2023-12-12T09:52:12.104813+0000","last_undegraded":"2023-12-12T09:52:12.104813+0000","last_fullsized":"2023-12-12T09:52:12.104813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:47.907544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.059506+0000","last_change":"2023-12-12T09:52:08.030292+0000","last_active":"2023-12-12T09:52:12.059506+0000","last_peered":"2023-12-12T09:52:12.059506+0000","last_clean":"2023-12-12T09:52:12.059506+0000","last_became_active":"2023-12-12T09:52:08.029117+0000","last_became_peered":"2023-12-12T09:52:08.029117+0000","last_unstale":"2023-12-12T09:52:12.059506+0000","last_undegraded":"2023-12-12T09:52:12.059506+0000","last_fullsized":"2023-12-12T09:52:12.059506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:00.775590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103702+0000","last_change":"2023-12-12T09:52:08.040587+0000","last_active":"2023-12-12T09:52:12.103702+0000","last_peered":"2023-12-12T09:52:12.103702+0000","last_clean":"2023-12-12T09:52:12.103702+0000","last_became_active":"2023-12-12T09:52:08.040204+0000","last_became_peered":"2023-12-12T09:52:08.040204+0000","last_unstale":"2023-12-12T09:52:12.103702+0000","last_undegraded":"2023-12-12T09:52:12.103702+0000","last_fullsized":"2023-12-12T09:52:12.103702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:44:35.253829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.073636+0000","last_change":"2023-12-12T09:52:08.015126+0000","last_active":"2023-12-12T09:52:12.073636+0000","last_peered":"2023-12-12T09:52:12.073636+0000","last_clean":"2023-12-12T09:52:12.073636+0000","last_became_active":"2023-12-12T09:52:08.014817+0000","last_became_peered":"2023-12-12T09:52:08.014817+0000","last_unstale":"2023-12-12T09:52:12.073636+0000","last_undegraded":"2023-12-12T09:52:12.073636+0000","last_fullsized":"2023-12-12T09:52:12.073636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.103966+0000","last_change":"2023-12-12T09:52:08.034136+0000","last_active":"2023-12-12T09:52:12.103966+0000","last_peered":"2023-12-12T09:52:12.103966+0000","last_clean":"2023-12-12T09:52:12.103966+0000","last_became_active":"2023-12-12T09:52:08.033810+0000","last_became_peered":"2023-12-12T09:52:08.033810+0000","last_unstale":"2023-12-12T09:52:12.103966+0000","last_undegraded":"2023-12-12T09:52:12.103966+0000","last_fullsized":"2023-12-12T09:52:12.103966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.950798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.074787+0000","last_change":"2023-12-12T09:52:08.032774+0000","last_active":"2023-12-12T09:52:12.074787+0000","last_peered":"2023-12-12T09:52:12.074787+0000","last_clean":"2023-12-12T09:52:12.074787+0000","last_became_active":"2023-12-12T09:52:08.030758+0000","last_became_peered":"2023-12-12T09:52:08.030758+0000","last_unstale":"2023-12-12T09:52:12.074787+0000","last_undegraded":"2023-12-12T09:52:12.074787+0000","last_fullsized":"2023-12-12T09:52:12.074787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:33:57.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.065632+0000","last_change":"2023-12-12T09:52:08.032385+0000","last_active":"2023-12-12T09:52:12.065632+0000","last_peered":"2023-12-12T09:52:12.065632+0000","last_clean":"2023-12-12T09:52:12.065632+0000","last_became_active":"2023-12-12T09:52:08.032067+0000","last_became_peered":"2023-12-12T09:52:08.032067+0000","last_unstale":"2023-12-12T09:52:12.065632+0000","last_undegraded":"2023-12-12T09:52:12.065632+0000","last_fullsized":"2023-12-12T09:52:12.065632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.015623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.056995+0000","last_change":"2023-12-12T09:52:08.028105+0000","last_active":"2023-12-12T09:52:12.056995+0000","last_peered":"2023-12-12T09:52:12.056995+0000","last_clean":"2023-12-12T09:52:12.056995+0000","last_became_active":"2023-12-12T09:52:08.027786+0000","last_became_peered":"2023-12-12T09:52:08.027786+0000","last_unstale":"2023-12-12T09:52:12.056995+0000","last_undegraded":"2023-12-12T09:52:12.056995+0000","last_fullsized":"2023-12-12T09:52:12.056995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:40:58.571743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-12T09:52:12.104765+0000","last_change":"2023-12-12T09:52:08.034061+0000","last_active":"2023-12-12T09:52:12.104765+0000","last_peered":"2023-12-12T09:52:12.104765+0000","last_clean":"2023-12-12T09:52:12.104765+0000","last_became_active":"2023-12-12T09:52:08.033745+0000","last_became_peered":"2023-12-12T09:52:08.033745+0000","last_unstale":"2023-12-12T09:52:12.104765+0000","last_undegraded":"2023-12-12T09:52:12.104765+0000","last_fullsized":"2023-12-12T09:52:12.104765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:06.956742+0000","last_clean_scrub_stamp":"2023-12-12T09:52:06.956742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:00:51.434174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4637,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7970000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2749999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7669999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.821}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8720000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":98899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5840000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":98933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8279999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8690000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.738}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.823}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.653}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:52:43.691 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-12T09:52:43.691 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-12T09:52:43.691 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-12T09:52:43.692 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-12T09:52:43.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.930+0000 7f0a6a432640 1 Processor -- start 2023-12-12T09:52:43.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.930+0000 7f0a6a432640 1 -- start start 2023-12-12T09:52:43.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.931+0000 7f0a6a432640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0a641439c0 0x7f0a640cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.931+0000 7f0a6a432640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a6406cd70 0x7f0a640c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.931+0000 7f0a6a432640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a640c9600 con 0x7f0a64143da0 2023-12-12T09:52:43.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.931+0000 7f0a6a432640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6406c8e0 con 0x7f0a641439c0 2023-12-12T09:52:43.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.931+0000 7f0a6a432640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a640cb140 con 0x7f0a6406cd70 2023-12-12T09:52:43.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.932+0000 7f0a69430640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0a641439c0 0x7f0a640cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.932+0000 7f0a69430640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0a641439c0 0x7f0a640cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40676/0 (socket says 172.21.15.72:40676) 2023-12-12T09:52:43.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.932+0000 7f0a69430640 1 -- 172.21.15.72:0/3223766865 learned_addr learned my addr 172.21.15.72:0/3223766865 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:43.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.932+0000 7f0a69c31640 1 --2- 172.21.15.72:0/3223766865 >> v2:172.21.15.113:3300/0 conn(0x7f0a6406cd70 0x7f0a640c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.932+0000 7f0a69430640 1 -- 172.21.15.72:0/3223766865 >> v2:172.21.15.113:3300/0 conn(0x7f0a6406cd70 msgr2=0x7f0a640c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:43.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.932+0000 7f0a69430640 1 --2- 172.21.15.72:0/3223766865 >> v2:172.21.15.113:3300/0 conn(0x7f0a6406cd70 0x7f0a640c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.932+0000 7f0a69430640 1 -- 172.21.15.72:0/3223766865 >> v1:172.21.15.113:6789/0 conn(0x7f0a64143da0 legacy=0x7f0a640cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:43.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.932+0000 7f0a69430640 1 -- 172.21.15.72:0/3223766865 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6406bfb0 con 0x7f0a641439c0 2023-12-12T09:52:43.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.934+0000 7f0a69430640 1 --2- 172.21.15.72:0/3223766865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a640cab40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a6406d3e0 tx=0x7f0a6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52b69c2b5ae7475 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:43.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.934+0000 7f0a5bfff640 1 -- 172.21.15.72:0/3223766865 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60031870 con 0x7f0a641439c0 2023-12-12T09:52:43.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a5bfff640 1 -- 172.21.15.72:0/3223766865 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a600319d0 con 0x7f0a641439c0 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a5bfff640 1 -- 172.21.15.72:0/3223766865 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60031d60 con 0x7f0a641439c0 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a6a432640 1 -- 172.21.15.72:0/3223766865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 msgr2=0x7f0a640cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a6a432640 1 --2- 172.21.15.72:0/3223766865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a640cab40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a6406d3e0 tx=0x7f0a6002b910 comp rx=0 tx=0).stop 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a6a432640 1 -- 172.21.15.72:0/3223766865 shutdown_connections 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a6a432640 1 --2- 172.21.15.72:0/3223766865 >> v2:172.21.15.113:3300/0 conn(0x7f0a6406cd70 0x7f0a640c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a6a432640 1 --2- 172.21.15.72:0/3223766865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a640cab40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a6a432640 1 -- 172.21.15.72:0/3223766865 >> 172.21.15.72:0/3223766865 conn(0x7f0a640b06c0 msgr2=0x7f0a640b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a6a432640 1 -- 172.21.15.72:0/3223766865 shutdown_connections 2023-12-12T09:52:43.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.935+0000 7f0a6a432640 1 -- 172.21.15.72:0/3223766865 wait complete. 2023-12-12T09:52:43.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.937+0000 7f0a6a432640 1 Processor -- start 2023-12-12T09:52:43.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.937+0000 7f0a6a432640 1 -- start start 2023-12-12T09:52:43.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.938+0000 7f0a6a432640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6406cd70 0x7f0a64140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.938+0000 7f0a6a432640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a64159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.938+0000 7f0a6a432640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a640b3830 0x7f0a640ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.938+0000 7f0a6a432640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640b44f0 con 0x7f0a641439c0 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.938+0000 7f0a6a432640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6413fd80 con 0x7f0a640b3830 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.938+0000 7f0a6a432640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640b4210 con 0x7f0a6406cd70 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.938+0000 7f0a68c2f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a64159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.938+0000 7f0a68c2f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a64159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40688/0 (socket says 172.21.15.72:40688) 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a68c2f640 1 -- 172.21.15.72:0/330592149 learned_addr learned my addr 172.21.15.72:0/330592149 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a68c2f640 1 -- 172.21.15.72:0/330592149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6406cd70 msgr2=0x7f0a64140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a69430640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6406cd70 0x7f0a64140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a69c31640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a640b3830 0x7f0a640ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a68c2f640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6406cd70 0x7f0a64140820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a68c2f640 1 -- 172.21.15.72:0/330592149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a640b3830 msgr2=0x7f0a640ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a68c2f640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a640b3830 0x7f0a640ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a68c2f640 1 -- 172.21.15.72:0/330592149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a600094d0 con 0x7f0a641439c0 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a68c2f640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a64159b60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c400 tx=0x7f0a5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a59ffb640 1 -- 172.21.15.72:0/330592149 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54011790 con 0x7f0a641439c0 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a59ffb640 1 -- 172.21.15.72:0/330592149 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a54011930 con 0x7f0a641439c0 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a59ffb640 1 -- 172.21.15.72:0/330592149 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54011ce0 con 0x7f0a641439c0 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.939+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a64140020 con 0x7f0a641439c0 2023-12-12T09:52:43.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.940+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6415a0a0 con 0x7f0a641439c0 2023-12-12T09:52:43.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.941+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6415a260 con 0x7f0a641439c0 2023-12-12T09:52:43.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.942+0000 7f0a59ffb640 1 -- 172.21.15.72:0/330592149 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a5401c460 con 0x7f0a641439c0 2023-12-12T09:52:43.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.942+0000 7f0a59ffb640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:43.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.943+0000 7f0a59ffb640 1 -- 172.21.15.72:0/330592149 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a54089850 con 0x7f0a641439c0 2023-12-12T09:52:43.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.943+0000 7f0a69430640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:43.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.943+0000 7f0a69430640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0a4806ae70 0x7f0a4806d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a60036000 tx=0x7f0a6003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:43.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:43.947+0000 7f0a59ffb640 1 -- 172.21.15.72:0/330592149 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a54054600 con 0x7f0a641439c0 2023-12-12T09:52:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.166+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0a64140540 con 0x7f0a641439c0 2023-12-12T09:52:44.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.166+0000 7f0a59ffb640 1 -- 172.21.15.72:0/330592149 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0a64140540 con 0x7f0a641439c0 2023-12-12T09:52:44.168 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:44.168 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-12T09:52:44.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0a4806ae70 msgr2=0x7f0a4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0a4806ae70 0x7f0a4806d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a60036000 tx=0x7f0a6003c040 comp rx=0 tx=0).stop 2023-12-12T09:52:44.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 msgr2=0x7f0a64159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a64159b60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c400 tx=0x7f0a5400c8c0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 shutdown_connections 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a640b3830 0x7f0a640ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a641439c0 0x7f0a64159b60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 --2- 172.21.15.72:0/330592149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6406cd70 0x7f0a64140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 >> 172.21.15.72:0/330592149 conn(0x7f0a640b06c0 msgr2=0x7f0a6406ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 shutdown_connections 2023-12-12T09:52:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.169+0000 7f0a6a432640 1 -- 172.21.15.72:0/330592149 wait complete. 2023-12-12T09:52:44.187 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-12T09:52:44.188 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:44.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.390+0000 7f646524b640 1 Processor -- start 2023-12-12T09:52:44.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.390+0000 7f646524b640 1 -- start start 2023-12-12T09:52:44.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.392+0000 7f646524b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f646015f260 0x7f64600ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.392+0000 7f646524b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64600cde40 con 0x7f64601626b0 2023-12-12T09:52:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.392+0000 7f646524b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6460162b40 con 0x7f64600708d0 2023-12-12T09:52:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.392+0000 7f646524b640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64600cd040 con 0x7f646015f260 2023-12-12T09:52:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.392+0000 7f645ed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f646015f260 0x7f64600ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.392+0000 7f645e59c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f64601626b0 0x7f64600701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52498/0 (socket says 172.21.15.72:52498) 2023-12-12T09:52:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.392+0000 7f645e59c640 1 -- 172.21.15.72:0/2104527517 learned_addr learned my addr 172.21.15.72:0/2104527517 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.393+0000 7f645dd9b640 1 -- 172.21.15.72:0/2104527517 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366867715 0 0) 0x7f64600cde40 con 0x7f64601626b0 2023-12-12T09:52:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.393+0000 7f645dd9b640 1 -- 172.21.15.72:0/2104527517 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f644c003520 con 0x7f64601626b0 2023-12-12T09:52:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.393+0000 7f645ed9d640 1 -- 172.21.15.72:0/2104527517 >> v1:172.21.15.113:6789/0 conn(0x7f64600708d0 legacy=0x7f646016b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.393+0000 7f645ed9d640 1 -- 172.21.15.72:0/2104527517 >> v1:172.21.15.72:6789/0 conn(0x7f64601626b0 legacy=0x7f64600701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.393+0000 7f645ed9d640 1 -- 172.21.15.72:0/2104527517 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64600cd230 con 0x7f646015f260 2023-12-12T09:52:44.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.394+0000 7f645ed9d640 1 --2- 172.21.15.72:0/2104527517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f64600ccb00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6458009d90 tx=0x7f645802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd039fb1fde929d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:44.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.395+0000 7f645dd9b640 1 -- 172.21.15.72:0/2104527517 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645802cd00 con 0x7f646015f260 2023-12-12T09:52:44.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.395+0000 7f645dd9b640 1 -- 172.21.15.72:0/2104527517 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645802ce60 con 0x7f646015f260 2023-12-12T09:52:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.395+0000 7f645dd9b640 1 -- 172.21.15.72:0/2104527517 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458034660 con 0x7f646015f260 2023-12-12T09:52:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.395+0000 7f646524b640 1 -- 172.21.15.72:0/2104527517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 msgr2=0x7f64600ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.395+0000 7f646524b640 1 --2- 172.21.15.72:0/2104527517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f64600ccb00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6458009d90 tx=0x7f645802bf50 comp rx=0 tx=0).stop 2023-12-12T09:52:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.395+0000 7f646524b640 1 -- 172.21.15.72:0/2104527517 shutdown_connections 2023-12-12T09:52:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.395+0000 7f646524b640 1 --2- 172.21.15.72:0/2104527517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f64600ccb00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.396+0000 7f646524b640 1 -- 172.21.15.72:0/2104527517 >> 172.21.15.72:0/2104527517 conn(0x7f64600b0a90 msgr2=0x7f64600b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.396+0000 7f646524b640 1 -- 172.21.15.72:0/2104527517 shutdown_connections 2023-12-12T09:52:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.396+0000 7f646524b640 1 -- 172.21.15.72:0/2104527517 wait complete. 2023-12-12T09:52:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.397+0000 7f646524b640 1 Processor -- start 2023-12-12T09:52:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.397+0000 7f646524b640 1 -- start start 2023-12-12T09:52:44.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.398+0000 7f646524b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64600708d0 0x7f64600b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.398+0000 7f646524b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f646015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.398+0000 7f646524b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64601626b0 0x7f646015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.398+0000 7f646524b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64600b2fe0 con 0x7f646015f260 2023-12-12T09:52:44.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.398+0000 7f646524b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460157f20 con 0x7f64600708d0 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.398+0000 7f646524b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64600b2740 con 0x7f64601626b0 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f646015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f646015a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40720/0 (socket says 172.21.15.72:40720) 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 -- 172.21.15.72:0/2233141675 learned_addr learned my addr 172.21.15.72:0/2233141675 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645ed9d640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64600708d0 0x7f64600b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645f59e640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64601626b0 0x7f646015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 -- 172.21.15.72:0/2233141675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64601626b0 msgr2=0x7f646015aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64601626b0 0x7f646015aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 -- 172.21.15.72:0/2233141675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64600708d0 msgr2=0x7f64600b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64600708d0 0x7f64600b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 -- 172.21.15.72:0/2233141675 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6458009a70 con 0x7f646015f260 2023-12-12T09:52:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.399+0000 7f645e59c640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f646015a530 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f644800c410 tx=0x7f644800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.400+0000 7f64577fe640 1 -- 172.21.15.72:0/2233141675 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448005ce0 con 0x7f646015f260 2023-12-12T09:52:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.400+0000 7f64577fe640 1 -- 172.21.15.72:0/2233141675 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6448005e40 con 0x7f646015f260 2023-12-12T09:52:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.400+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64600b2a20 con 0x7f646015f260 2023-12-12T09:52:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.400+0000 7f64577fe640 1 -- 172.21.15.72:0/2233141675 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480072f0 con 0x7f646015f260 2023-12-12T09:52:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.400+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64600b23c0 con 0x7f646015f260 2023-12-12T09:52:44.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.401+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6460157d80 con 0x7f646015f260 2023-12-12T09:52:44.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.402+0000 7f64577fe640 1 -- 172.21.15.72:0/2233141675 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6448007450 con 0x7f646015f260 2023-12-12T09:52:44.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.403+0000 7f64577fe640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f644006ae70 0x7f644006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.403+0000 7f64577fe640 1 -- 172.21.15.72:0/2233141675 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6448088ef0 con 0x7f646015f260 2023-12-12T09:52:44.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.403+0000 7f645ed9d640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f644006ae70 0x7f644006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.404+0000 7f645ed9d640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f644006ae70 0x7f644006d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6458036000 tx=0x7f6458030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.408+0000 7f64577fe640 1 -- 172.21.15.72:0/2233141675 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6460157d80 con 0x7f646015f260 2023-12-12T09:52:44.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.600+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6460157d80 con 0x7f646015f260 2023-12-12T09:52:44.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.603+0000 7f64577fe640 1 -- 172.21.15.72:0/2233141675 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f6460157d80 con 0x7f646015f260 2023-12-12T09:52:44.605 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:44.606 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:44.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f644006ae70 msgr2=0x7f644006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f644006ae70 0x7f644006d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6458036000 tx=0x7f6458030000 comp rx=0 tx=0).stop 2023-12-12T09:52:44.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 msgr2=0x7f646015a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f646015a530 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f644800c410 tx=0x7f644800c8d0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 shutdown_connections 2023-12-12T09:52:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f644006ae70 0x7f644006d330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64601626b0 0x7f646015aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f646015f260 0x7f646015a530 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 --2- 172.21.15.72:0/2233141675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64600708d0 0x7f64600b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.606+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 >> 172.21.15.72:0/2233141675 conn(0x7f64600b0a90 msgr2=0x7f646006c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.607+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 shutdown_connections 2023-12-12T09:52:44.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.607+0000 7f646524b640 1 -- 172.21.15.72:0/2233141675 wait complete. 2023-12-12T09:52:44.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:52:44.628 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:52:44.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.838+0000 7f104101a640 1 Processor -- start 2023-12-12T09:52:44.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.838+0000 7f104101a640 1 -- start start 2023-12-12T09:52:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.839+0000 7f104101a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f103c1543f0 0x7f103c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.839+0000 7f104101a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f103c1547d0 0x7f103c070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.839+0000 7f104101a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c0b3810 0x7f103c0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.839+0000 7f104101a640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103c15ae30 con 0x7f103c1547d0 2023-12-12T09:52:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.839+0000 7f104101a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103c15b110 con 0x7f103c1543f0 2023-12-12T09:52:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.839+0000 7f104101a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103c15b3f0 con 0x7f103c0b3810 2023-12-12T09:52:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1033fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f103c1547d0 0x7f103c070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1033fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f103c1547d0 0x7f103c070070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40730/0 (socket says 172.21.15.72:40730) 2023-12-12T09:52:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1033fff640 1 -- 172.21.15.72:0/2476648117 learned_addr learned my addr 172.21.15.72:0/2476648117 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f103bfff640 1 --2- 172.21.15.72:0/2476648117 >> v2:172.21.15.113:3300/0 conn(0x7f103c1543f0 0x7f103c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1040819640 1 --2- 172.21.15.72:0/2476648117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c0b3810 0x7f103c0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1033fff640 1 -- 172.21.15.72:0/2476648117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c0b3810 msgr2=0x7f103c0cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1033fff640 1 --2- 172.21.15.72:0/2476648117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c0b3810 0x7f103c0cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1033fff640 1 -- 172.21.15.72:0/2476648117 >> v2:172.21.15.113:3300/0 conn(0x7f103c1543f0 msgr2=0x7f103c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1033fff640 1 --2- 172.21.15.72:0/2476648117 >> v2:172.21.15.113:3300/0 conn(0x7f103c1543f0 0x7f103c15a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.840+0000 7f1033fff640 1 -- 172.21.15.72:0/2476648117 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0aca50 con 0x7f103c1547d0 2023-12-12T09:52:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.841+0000 7f1033fff640 1 --2- 172.21.15.72:0/2476648117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c1547d0 0x7f103c070070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f103c0cfcb0 tx=0x7f102402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816c1ff63d9e6f68 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f103b7fe640 1 -- 172.21.15.72:0/2476648117 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024031870 con 0x7f103c1547d0 2023-12-12T09:52:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f103b7fe640 1 -- 172.21.15.72:0/2476648117 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10240319d0 con 0x7f103c1547d0 2023-12-12T09:52:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f103b7fe640 1 -- 172.21.15.72:0/2476648117 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024031d60 con 0x7f103c1547d0 2023-12-12T09:52:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 -- 172.21.15.72:0/2476648117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c1547d0 msgr2=0x7f103c070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 --2- 172.21.15.72:0/2476648117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c1547d0 0x7f103c070070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f103c0cfcb0 tx=0x7f102402b910 comp rx=0 tx=0).stop 2023-12-12T09:52:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 -- 172.21.15.72:0/2476648117 shutdown_connections 2023-12-12T09:52:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 --2- 172.21.15.72:0/2476648117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c0b3810 0x7f103c0cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 --2- 172.21.15.72:0/2476648117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c1547d0 0x7f103c070070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 --2- 172.21.15.72:0/2476648117 >> v2:172.21.15.113:3300/0 conn(0x7f103c1543f0 0x7f103c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 -- 172.21.15.72:0/2476648117 >> 172.21.15.72:0/2476648117 conn(0x7f103c06a640 msgr2=0x7f103c06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 -- 172.21.15.72:0/2476648117 shutdown_connections 2023-12-12T09:52:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.842+0000 7f104101a640 1 -- 172.21.15.72:0/2476648117 wait complete. 2023-12-12T09:52:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.844+0000 7f104101a640 1 Processor -- start 2023-12-12T09:52:44.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.844+0000 7f104101a640 1 -- start start 2023-12-12T09:52:44.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f104101a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c0b3810 0x7f103c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f104101a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c1543f0 0x7f103c14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f104101a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103c1547d0 0x7f103c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f104101a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c15baa0 con 0x7f103c0b3810 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f104101a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c0aec40 con 0x7f103c1547d0 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f104101a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103c0aef20 con 0x7f103c1543f0 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f103bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c0b3810 0x7f103c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f103bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c0b3810 0x7f103c15d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40738/0 (socket says 172.21.15.72:40738) 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.845+0000 7f103bfff640 1 -- 172.21.15.72:0/4076561142 learned_addr learned my addr 172.21.15.72:0/4076561142 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f1033fff640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c1543f0 0x7f103c14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f1040819640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103c1547d0 0x7f103c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f103bfff640 1 -- 172.21.15.72:0/4076561142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c1543f0 msgr2=0x7f103c14e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f103bfff640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c1543f0 0x7f103c14e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f103bfff640 1 -- 172.21.15.72:0/4076561142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103c1547d0 msgr2=0x7f103c15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f103bfff640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103c1547d0 0x7f103c15bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:44.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f103bfff640 1 -- 172.21.15.72:0/4076561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240094d0 con 0x7f103c0b3810 2023-12-12T09:52:44.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f103bfff640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c0b3810 0x7f103c15d2d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f102c00c7f0 tx=0x7f102c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:44.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f10397fa640 1 -- 172.21.15.72:0/4076561142 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c004220 con 0x7f103c0b3810 2023-12-12T09:52:44.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f10397fa640 1 -- 172.21.15.72:0/4076561142 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c0043a0 con 0x7f103c0b3810 2023-12-12T09:52:44.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c0af200 con 0x7f103c0b3810 2023-12-12T09:52:44.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.846+0000 7f10397fa640 1 -- 172.21.15.72:0/4076561142 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c007860 con 0x7f103c0b3810 2023-12-12T09:52:44.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.847+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c15d810 con 0x7f103c0b3810 2023-12-12T09:52:44.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.848+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103c074450 con 0x7f103c0b3810 2023-12-12T09:52:44.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.848+0000 7f10397fa640 1 -- 172.21.15.72:0/4076561142 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f102c007bd0 con 0x7f103c0b3810 2023-12-12T09:52:44.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.849+0000 7f10397fa640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f101c06ae40 0x7f101c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:44.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.849+0000 7f10397fa640 1 -- 172.21.15.72:0/4076561142 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f102c089930 con 0x7f103c0b3810 2023-12-12T09:52:44.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.850+0000 7f1033fff640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f101c06ae40 0x7f101c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:44.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.850+0000 7f1033fff640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f101c06ae40 0x7f101c06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1024036000 tx=0x7f102403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:44.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:44.853+0000 7f10397fa640 1 -- 172.21.15.72:0/4076561142 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f102c0546f0 con 0x7f103c0b3810 2023-12-12T09:52:45.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.036+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f103c0b01e0 con 0x7f103c0b3810 2023-12-12T09:52:45.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.037+0000 7f10397fa640 1 -- 172.21.15.72:0/4076561142 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f102c01e0a0 con 0x7f103c0b3810 2023-12-12T09:52:45.039 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.039+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f101c06ae40 msgr2=0x7f101c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.039+0000 7f104101a640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f101c06ae40 0x7f101c06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1024036000 tx=0x7f102403c040 comp rx=0 tx=0).stop 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.039+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c0b3810 msgr2=0x7f103c15d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.039+0000 7f104101a640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c0b3810 0x7f103c15d2d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f102c00c7f0 tx=0x7f102c00ccb0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.040+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 shutdown_connections 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.040+0000 7f104101a640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f101c06ae40 0x7f101c06d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.040+0000 7f104101a640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f103c1547d0 0x7f103c15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.040+0000 7f104101a640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f103c1543f0 0x7f103c14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.040+0000 7f104101a640 1 --2- 172.21.15.72:0/4076561142 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f103c0b3810 0x7f103c15d2d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.040+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 >> 172.21.15.72:0/4076561142 conn(0x7f103c06a640 msgr2=0x7f103c1537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.040+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 shutdown_connections 2023-12-12T09:52:45.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.040+0000 7f104101a640 1 -- 172.21.15.72:0/4076561142 wait complete. 2023-12-12T09:52:45.059 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:52:45.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.270+0000 7ff826305640 1 Processor -- start 2023-12-12T09:52:45.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.270+0000 7ff826305640 1 -- start start 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.271+0000 7ff826305640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff82015d4e0 0x7ff8200ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.271+0000 7ff826305640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff820071a90 0x7ff82006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.271+0000 7ff826305640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8200cad40 con 0x7ff8200ca8e0 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.271+0000 7ff826305640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff820071ee0 con 0x7ff82015d4e0 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.271+0000 7ff826305640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82015d8c0 con 0x7ff820071a90 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.272+0000 7ff81ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff82015d4e0 0x7ff8200ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.272+0000 7ff81ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff82015d4e0 0x7ff8200ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58946/0 (socket says 172.21.15.72:58946) 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.272+0000 7ff81ffff640 1 -- 172.21.15.72:0/86578113 learned_addr learned my addr 172.21.15.72:0/86578113 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.272+0000 7ff8248a2640 1 --2- 172.21.15.72:0/86578113 >> v2:172.21.15.113:3300/0 conn(0x7ff820071a90 0x7ff82006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.272+0000 7ff81ffff640 1 -- 172.21.15.72:0/86578113 >> v2:172.21.15.113:3300/0 conn(0x7ff820071a90 msgr2=0x7ff82006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.272+0000 7ff81ffff640 1 --2- 172.21.15.72:0/86578113 >> v2:172.21.15.113:3300/0 conn(0x7ff820071a90 0x7ff82006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.272+0000 7ff81ffff640 1 -- 172.21.15.72:0/86578113 >> v1:172.21.15.113:6789/0 conn(0x7ff8200ca8e0 legacy=0x7ff820071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:45.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.272+0000 7ff81ffff640 1 -- 172.21.15.72:0/86578113 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82006d870 con 0x7ff82015d4e0 2023-12-12T09:52:45.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.273+0000 7ff81ffff640 1 --2- 172.21.15.72:0/86578113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff82015d4e0 0x7ff8200ca3a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff8200cfd00 tx=0x7ff80802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d227f6f5330ad55 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:45.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff81effd640 1 -- 172.21.15.72:0/86578113 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8080317e0 con 0x7ff82015d4e0 2023-12-12T09:52:45.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff81effd640 1 -- 172.21.15.72:0/86578113 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff808031940 con 0x7ff82015d4e0 2023-12-12T09:52:45.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff81effd640 1 -- 172.21.15.72:0/86578113 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808031cd0 con 0x7ff82015d4e0 2023-12-12T09:52:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff826305640 1 -- 172.21.15.72:0/86578113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff82015d4e0 msgr2=0x7ff8200ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff826305640 1 --2- 172.21.15.72:0/86578113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff82015d4e0 0x7ff8200ca3a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff8200cfd00 tx=0x7ff80802b880 comp rx=0 tx=0).stop 2023-12-12T09:52:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff826305640 1 -- 172.21.15.72:0/86578113 shutdown_connections 2023-12-12T09:52:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff826305640 1 --2- 172.21.15.72:0/86578113 >> v2:172.21.15.113:3300/0 conn(0x7ff820071a90 0x7ff82006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff826305640 1 --2- 172.21.15.72:0/86578113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff82015d4e0 0x7ff8200ca3a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.274+0000 7ff826305640 1 -- 172.21.15.72:0/86578113 >> 172.21.15.72:0/86578113 conn(0x7ff8200cc7d0 msgr2=0x7ff8200ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.275+0000 7ff826305640 1 -- 172.21.15.72:0/86578113 shutdown_connections 2023-12-12T09:52:45.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.275+0000 7ff826305640 1 -- 172.21.15.72:0/86578113 wait complete. 2023-12-12T09:52:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.276+0000 7ff826305640 1 Processor -- start 2023-12-12T09:52:45.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.276+0000 7ff826305640 1 -- start start 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.277+0000 7ff826305640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff820071a90 0x7ff8201506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.277+0000 7ff826305640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8200ca8e0 0x7ff8200c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.277+0000 7ff826305640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff82015d4e0 0x7ff8200ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.277+0000 7ff826305640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200ad1b0 con 0x7ff8200ca8e0 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.277+0000 7ff826305640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820150c10 con 0x7ff82015d4e0 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.277+0000 7ff826305640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200c8f50 con 0x7ff820071a90 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8200ca8e0 0x7ff8200c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8200ca8e0 0x7ff8200c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58954/0 (socket says 172.21.15.72:58954) 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 -- 172.21.15.72:0/312531479 learned_addr learned my addr 172.21.15.72:0/312531479 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:45.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81ffff640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff820071a90 0x7ff8201506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff8248a2640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff82015d4e0 0x7ff8200ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 -- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff820071a90 msgr2=0x7ff8201506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff820071a90 0x7ff8201506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 -- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff82015d4e0 msgr2=0x7ff8200ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff82015d4e0 0x7ff8200ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 -- 172.21.15.72:0/312531479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080094d0 con 0x7ff8200ca8e0 2023-12-12T09:52:45.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81ffff640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff820071a90 0x7ff8201506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:45.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.278+0000 7ff81f7fe640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8200ca8e0 0x7ff8200c8720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff81000ba50 tx=0x7ff81000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:45.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.279+0000 7ff81cff9640 1 -- 172.21.15.72:0/312531479 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000cbe0 con 0x7ff8200ca8e0 2023-12-12T09:52:45.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.279+0000 7ff81cff9640 1 -- 172.21.15.72:0/312531479 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81000cd80 con 0x7ff8200ca8e0 2023-12-12T09:52:45.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.279+0000 7ff81cff9640 1 -- 172.21.15.72:0/312531479 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810014680 con 0x7ff8200ca8e0 2023-12-12T09:52:45.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.279+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8200c9230 con 0x7ff8200ca8e0 2023-12-12T09:52:45.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.279+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8200c8c60 con 0x7ff8200ca8e0 2023-12-12T09:52:45.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.280+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8200745d0 con 0x7ff8200ca8e0 2023-12-12T09:52:45.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.281+0000 7ff81cff9640 1 -- 172.21.15.72:0/312531479 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff8100147e0 con 0x7ff8200ca8e0 2023-12-12T09:52:45.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.282+0000 7ff81cff9640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff80006ae40 0x7ff80006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.282+0000 7ff81cff9640 1 -- 172.21.15.72:0/312531479 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff810089730 con 0x7ff8200ca8e0 2023-12-12T09:52:45.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.282+0000 7ff81ffff640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff80006ae40 0x7ff80006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.283+0000 7ff81ffff640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff80006ae40 0x7ff80006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff808036000 tx=0x7ff80803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:45.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.286+0000 7ff81cff9640 1 -- 172.21.15.72:0/312531479 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff810054630 con 0x7ff8200ca8e0 2023-12-12T09:52:45.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.474+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8200ca100 con 0x7ff8200ca8e0 2023-12-12T09:52:45.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.475+0000 7ff81cff9640 1 -- 172.21.15.72:0/312531479 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff810057c50 con 0x7ff8200ca8e0 2023-12-12T09:52:45.477 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff80006ae40 msgr2=0x7ff80006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff80006ae40 0x7ff80006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff808036000 tx=0x7ff80803d040 comp rx=0 tx=0).stop 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8200ca8e0 msgr2=0x7ff8200c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8200ca8e0 0x7ff8200c8720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff81000ba50 tx=0x7ff81000bf10 comp rx=0 tx=0).stop 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 shutdown_connections 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff80006ae40 0x7ff80006d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff82015d4e0 0x7ff8200ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8200ca8e0 0x7ff8200c8720 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 --2- 172.21.15.72:0/312531479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff820071a90 0x7ff8201506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 >> 172.21.15.72:0/312531479 conn(0x7ff8200cc7d0 msgr2=0x7ff8200b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:45.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 shutdown_connections 2023-12-12T09:52:45.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.478+0000 7ff826305640 1 -- 172.21.15.72:0/312531479 wait complete. 2023-12-12T09:52:45.499 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.713+0000 7f974a41e640 1 Processor -- start 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.713+0000 7f974a41e640 1 -- start start 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.715+0000 7f974a41e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f97440ae300 0x7f974406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.715+0000 7f974a41e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97440cdaa0 con 0x7f974415a1e0 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.715+0000 7f974a41e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97440cdd80 con 0x7f974406ab70 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.715+0000 7f974a41e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97440ce060 con 0x7f97440ae300 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.715+0000 7f9743fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f974415a1e0 0x7f974406a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59272/0 (socket says 172.21.15.72:59272) 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.715+0000 7f9743fff640 1 -- 172.21.15.72:0/460123981 learned_addr learned my addr 172.21.15.72:0/460123981 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.715+0000 7f97489bb640 1 --2- 172.21.15.72:0/460123981 >> v2:172.21.15.72:3300/0 conn(0x7f97440ae300 0x7f974406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.716+0000 7f9742ffd640 1 -- 172.21.15.72:0/460123981 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216718396 0 0) 0x7f97440cdaa0 con 0x7f974415a1e0 2023-12-12T09:52:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.716+0000 7f97489bb640 1 -- 172.21.15.72:0/460123981 >> v1:172.21.15.113:6789/0 conn(0x7f974406ab70 legacy=0x7f974415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.716+0000 7f97489bb640 1 -- 172.21.15.72:0/460123981 >> v1:172.21.15.72:6789/0 conn(0x7f974415a1e0 legacy=0x7f974406a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.716+0000 7f97489bb640 1 -- 172.21.15.72:0/460123981 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97440ce340 con 0x7f97440ae300 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.717+0000 7f97489bb640 1 --2- 172.21.15.72:0/460123981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97440ae300 0x7f974406ca10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f973c009d90 tx=0x7f973c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cb9199f05d5c9d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.717+0000 7f9742ffd640 1 -- 172.21.15.72:0/460123981 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c02cb40 con 0x7f97440ae300 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.717+0000 7f9742ffd640 1 -- 172.21.15.72:0/460123981 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973c02cca0 con 0x7f97440ae300 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.717+0000 7f9742ffd640 1 -- 172.21.15.72:0/460123981 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c034660 con 0x7f97440ae300 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.717+0000 7f974a41e640 1 -- 172.21.15.72:0/460123981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97440ae300 msgr2=0x7f974406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.717+0000 7f974a41e640 1 --2- 172.21.15.72:0/460123981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97440ae300 0x7f974406ca10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f973c009d90 tx=0x7f973c02bd90 comp rx=0 tx=0).stop 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.718+0000 7f974a41e640 1 -- 172.21.15.72:0/460123981 shutdown_connections 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.718+0000 7f974a41e640 1 --2- 172.21.15.72:0/460123981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97440ae300 0x7f974406ca10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.718+0000 7f974a41e640 1 -- 172.21.15.72:0/460123981 >> 172.21.15.72:0/460123981 conn(0x7f97440cee70 msgr2=0x7f97440cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.718+0000 7f974a41e640 1 -- 172.21.15.72:0/460123981 shutdown_connections 2023-12-12T09:52:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.718+0000 7f974a41e640 1 -- 172.21.15.72:0/460123981 wait complete. 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.719+0000 7f974a41e640 1 Processor -- start 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.720+0000 7f974a41e640 1 -- start start 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f974a41e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f974406ab70 0x7f97440b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f974a41e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97440ae300 0x7f9744158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f974a41e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f974415a1e0 0x7f97440ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f974a41e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97440ace20 con 0x7f974406ab70 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f974a41e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974409aa30 con 0x7f97440ae300 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f974a41e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974409ad10 con 0x7f974415a1e0 2023-12-12T09:52:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f9743fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f974406ab70 0x7f97440b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f9743fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f974406ab70 0x7f97440b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58972/0 (socket says 172.21.15.72:58972) 2023-12-12T09:52:45.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f9743fff640 1 -- 172.21.15.72:0/718745255 learned_addr learned my addr 172.21.15.72:0/718745255 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:45.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f9743fff640 1 -- 172.21.15.72:0/718745255 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f974415a1e0 msgr2=0x7f97440ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f9743fff640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f974415a1e0 0x7f97440ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f9743fff640 1 -- 172.21.15.72:0/718745255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97440ae300 msgr2=0x7f9744158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f9743fff640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97440ae300 0x7f9744158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.721+0000 7f9743fff640 1 -- 172.21.15.72:0/718745255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c009a70 con 0x7f974406ab70 2023-12-12T09:52:45.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.722+0000 7f9743fff640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f974406ab70 0x7f97440b1930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f973400c430 tx=0x7f973400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.722+0000 7f9740ff9640 1 -- 172.21.15.72:0/718745255 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340074e0 con 0x7f974406ab70 2023-12-12T09:52:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.722+0000 7f9740ff9640 1 -- 172.21.15.72:0/718745255 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734007660 con 0x7f974406ab70 2023-12-12T09:52:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.722+0000 7f9740ff9640 1 -- 172.21.15.72:0/718745255 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734004a20 con 0x7f974406ab70 2023-12-12T09:52:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.722+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974409aff0 con 0x7f974406ab70 2023-12-12T09:52:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.722+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974409a650 con 0x7f974406ab70 2023-12-12T09:52:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.723+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744074450 con 0x7f974406ab70 2023-12-12T09:52:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.724+0000 7f9740ff9640 1 -- 172.21.15.72:0/718745255 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9734004b80 con 0x7f974406ab70 2023-12-12T09:52:45.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.725+0000 7f9740ff9640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f973006ae40 0x7f973006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:45.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.725+0000 7f9740ff9640 1 -- 172.21.15.72:0/718745255 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9734089710 con 0x7f974406ab70 2023-12-12T09:52:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.725+0000 7f97437fe640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f973006ae40 0x7f973006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.726+0000 7f97437fe640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f973006ae40 0x7f973006d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f97441f6c60 tx=0x7f97380079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:45.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.729+0000 7f9740ff9640 1 -- 172.21.15.72:0/718745255 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97340544d0 con 0x7f974406ab70 2023-12-12T09:52:45.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.919+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97440cc340 con 0x7f974406ab70 2023-12-12T09:52:45.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.920+0000 7f9740ff9640 1 -- 172.21.15.72:0/718745255 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9734057af0 con 0x7f974406ab70 2023-12-12T09:52:45.923 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:52:45.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.922+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f973006ae40 msgr2=0x7f973006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.922+0000 7f974a41e640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f973006ae40 0x7f973006d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f97441f6c60 tx=0x7f97380079b0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f974406ab70 msgr2=0x7f97440b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:45.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f974406ab70 0x7f97440b1930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f973400c430 tx=0x7f973400c8f0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 shutdown_connections 2023-12-12T09:52:45.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f973006ae40 0x7f973006d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f974415a1e0 0x7f97440ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97440ae300 0x7f9744158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 --2- 172.21.15.72:0/718745255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f974406ab70 0x7f97440b1930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:45.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 >> 172.21.15.72:0/718745255 conn(0x7f97440cee70 msgr2=0x7f97440cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:45.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 shutdown_connections 2023-12-12T09:52:45.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:45.923+0000 7f974a41e640 1 -- 172.21.15.72:0/718745255 wait complete. 2023-12-12T09:52:45.943 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:52:46.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.157+0000 7fc19ba00640 1 Processor -- start 2023-12-12T09:52:46.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.157+0000 7fc19ba00640 1 -- start start 2023-12-12T09:52:46.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc19ba00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1940b2f80 0x7fc19415e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc19ba00640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc194160f70 0x7fc1941613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc19ba00640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19415e0e0 con 0x7fc194070ee0 2023-12-12T09:52:46.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc19ba00640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1940b3360 con 0x7fc194160f70 2023-12-12T09:52:46.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc19ba00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc194161dc0 con 0x7fc1940b2f80 2023-12-12T09:52:46.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc198f9b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc194160f70 0x7fc1941613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc199f9d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc194070ee0 0x7fc19406a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59286/0 (socket says 172.21.15.72:59286) 2023-12-12T09:52:46.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc199f9d640 1 -- 172.21.15.72:0/3596162912 learned_addr learned my addr 172.21.15.72:0/3596162912 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:46.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.159+0000 7fc19979c640 1 --2- 172.21.15.72:0/3596162912 >> v2:172.21.15.113:3300/0 conn(0x7fc1940b2f80 0x7fc19415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.160+0000 7fc198f9b640 1 -- 172.21.15.72:0/3596162912 >> v2:172.21.15.113:3300/0 conn(0x7fc1940b2f80 msgr2=0x7fc19415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:46.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.160+0000 7fc198f9b640 1 --2- 172.21.15.72:0/3596162912 >> v2:172.21.15.113:3300/0 conn(0x7fc1940b2f80 0x7fc19415e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.160+0000 7fc198f9b640 1 -- 172.21.15.72:0/3596162912 >> v1:172.21.15.72:6789/0 conn(0x7fc194070ee0 legacy=0x7fc19406a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.160+0000 7fc198f9b640 1 -- 172.21.15.72:0/3596162912 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1940b2820 con 0x7fc194160f70 2023-12-12T09:52:46.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.160+0000 7fc18bfff640 1 -- 172.21.15.72:0/3596162912 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346137111 0 0) 0x7fc19415e0e0 con 0x7fc194070ee0 2023-12-12T09:52:46.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.161+0000 7fc198f9b640 1 --2- 172.21.15.72:0/3596162912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc194160f70 0x7fc1941613f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc194071ab0 tx=0x7fc18402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a65750d8c621c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.161+0000 7fc18bfff640 1 -- 172.21.15.72:0/3596162912 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18402e070 con 0x7fc194160f70 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.161+0000 7fc18bfff640 1 -- 172.21.15.72:0/3596162912 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1840315a0 con 0x7fc194160f70 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.161+0000 7fc18bfff640 1 -- 172.21.15.72:0/3596162912 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184031930 con 0x7fc194160f70 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.162+0000 7fc19ba00640 1 -- 172.21.15.72:0/3596162912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc194160f70 msgr2=0x7fc1941613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.162+0000 7fc19ba00640 1 --2- 172.21.15.72:0/3596162912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc194160f70 0x7fc1941613f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc194071ab0 tx=0x7fc18402b840 comp rx=0 tx=0).stop 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.162+0000 7fc19ba00640 1 -- 172.21.15.72:0/3596162912 shutdown_connections 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.162+0000 7fc19ba00640 1 --2- 172.21.15.72:0/3596162912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc194160f70 0x7fc1941613f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.162+0000 7fc19ba00640 1 --2- 172.21.15.72:0/3596162912 >> v2:172.21.15.113:3300/0 conn(0x7fc1940b2f80 0x7fc19415e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.162+0000 7fc19ba00640 1 -- 172.21.15.72:0/3596162912 >> 172.21.15.72:0/3596162912 conn(0x7fc1940b16b0 msgr2=0x7fc1940b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.162+0000 7fc19ba00640 1 -- 172.21.15.72:0/3596162912 shutdown_connections 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.162+0000 7fc19ba00640 1 -- 172.21.15.72:0/3596162912 wait complete. 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.164+0000 7fc19ba00640 1 Processor -- start 2023-12-12T09:52:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.164+0000 7fc19ba00640 1 -- start start 2023-12-12T09:52:46.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc19ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc194070ee0 0x7fc19414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc19ba00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1940b2f80 0x7fc19414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc19ba00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc194160f70 0x7fc1940b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc19ba00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19415d410 con 0x7fc1940b2f80 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc19ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19414a950 con 0x7fc194070ee0 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc19ba00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19414e210 con 0x7fc194160f70 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc198f9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1940b2f80 0x7fc19414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc198f9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1940b2f80 0x7fc19414dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59000/0 (socket says 172.21.15.72:59000) 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc198f9b640 1 -- 172.21.15.72:0/545368496 learned_addr learned my addr 172.21.15.72:0/545368496 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc199f9d640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc194160f70 0x7fc1940b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc19979c640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc194070ee0 0x7fc19414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc198f9b640 1 -- 172.21.15.72:0/545368496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc194160f70 msgr2=0x7fc1940b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc198f9b640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc194160f70 0x7fc1940b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc198f9b640 1 -- 172.21.15.72:0/545368496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc194070ee0 msgr2=0x7fc19414ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc198f9b640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc194070ee0 0x7fc19414ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.165+0000 7fc198f9b640 1 -- 172.21.15.72:0/545368496 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc184009400 con 0x7fc1940b2f80 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.166+0000 7fc198f9b640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1940b2f80 0x7fc19414dcd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc18402be50 tx=0x7fc1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.166+0000 7fc189ffb640 1 -- 172.21.15.72:0/545368496 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184032840 con 0x7fc1940b2f80 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.166+0000 7fc189ffb640 1 -- 172.21.15.72:0/545368496 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1840329c0 con 0x7fc1940b2f80 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.166+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1941f5e40 con 0x7fc1940b2f80 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.166+0000 7fc189ffb640 1 -- 172.21.15.72:0/545368496 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184032d70 con 0x7fc1940b2f80 2023-12-12T09:52:46.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.166+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1941f63e0 con 0x7fc1940b2f80 2023-12-12T09:52:46.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.167+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1940745d0 con 0x7fc1940b2f80 2023-12-12T09:52:46.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.168+0000 7fc189ffb640 1 -- 172.21.15.72:0/545368496 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc1840453f0 con 0x7fc1940b2f80 2023-12-12T09:52:46.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.169+0000 7fc189ffb640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc17806ae40 0x7fc17806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.169+0000 7fc189ffb640 1 -- 172.21.15.72:0/545368496 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc1840ab8c0 con 0x7fc1940b2f80 2023-12-12T09:52:46.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.169+0000 7fc19979c640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc17806ae40 0x7fc17806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.170+0000 7fc19979c640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc17806ae40 0x7fc17806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc190006fd0 tx=0x7fc190008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.173+0000 7fc189ffb640 1 -- 172.21.15.72:0/545368496 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc184076710 con 0x7fc1940b2f80 2023-12-12T09:52:46.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.361+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc19406b9a0 con 0x7fc1940b2f80 2023-12-12T09:52:46.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.362+0000 7fc189ffb640 1 -- 172.21.15.72:0/545368496 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc184079d30 con 0x7fc1940b2f80 2023-12-12T09:52:46.364 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:52:46.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.364+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc17806ae40 msgr2=0x7fc17806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.364+0000 7fc19ba00640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc17806ae40 0x7fc17806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc190006fd0 tx=0x7fc190008040 comp rx=0 tx=0).stop 2023-12-12T09:52:46.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.364+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1940b2f80 msgr2=0x7fc19414dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.364+0000 7fc19ba00640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1940b2f80 0x7fc19414dcd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc18402be50 tx=0x7fc1840046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.364+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 shutdown_connections 2023-12-12T09:52:46.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.364+0000 7fc19ba00640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc17806ae40 0x7fc17806d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.365+0000 7fc19ba00640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc194160f70 0x7fc1940b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.365+0000 7fc19ba00640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1940b2f80 0x7fc19414dcd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.365+0000 7fc19ba00640 1 --2- 172.21.15.72:0/545368496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc194070ee0 0x7fc19414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.365+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 >> 172.21.15.72:0/545368496 conn(0x7fc1940b16b0 msgr2=0x7fc1940b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:46.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.365+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 shutdown_connections 2023-12-12T09:52:46.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.365+0000 7fc19ba00640 1 -- 172.21.15.72:0/545368496 wait complete. 2023-12-12T09:52:46.384 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:52:46.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.574+0000 7fb3074b4640 1 Processor -- start 2023-12-12T09:52:46.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.575+0000 7fb3074b4640 1 -- start start 2023-12-12T09:52:46.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.576+0000 7fb3074b4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3001467c0 0x7fb3000cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.576+0000 7fb3074b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb30006cef0 0x7fb3000cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.576+0000 7fb3074b4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30006d510 con 0x7fb300146ba0 2023-12-12T09:52:46.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.576+0000 7fb3074b4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3000ac860 con 0x7fb3001467c0 2023-12-12T09:52:46.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.576+0000 7fb3074b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30006d7f0 con 0x7fb30006cef0 2023-12-12T09:52:46.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.577+0000 7fb305a51640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb30006cef0 0x7fb3000cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.577+0000 7fb305250640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3001467c0 0x7fb3000cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.577+0000 7fb305a51640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb30006cef0 0x7fb3000cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:34000/0 (socket says 172.21.15.72:34000) 2023-12-12T09:52:46.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.577+0000 7fb305a51640 1 -- 172.21.15.72:0/2879840762 learned_addr learned my addr 172.21.15.72:0/2879840762 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:46.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.577+0000 7fb305250640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3001467c0 0x7fb3000cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:53114/0 (socket says 172.21.15.72:53114) 2023-12-12T09:52:46.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.578+0000 7fb305a51640 1 -- 172.21.15.72:0/2879840762 >> v2:172.21.15.113:3300/0 conn(0x7fb3001467c0 msgr2=0x7fb3000cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:46.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.578+0000 7fb305a51640 1 --2- 172.21.15.72:0/2879840762 >> v2:172.21.15.113:3300/0 conn(0x7fb3001467c0 0x7fb3000cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.578+0000 7fb305a51640 1 -- 172.21.15.72:0/2879840762 >> v1:172.21.15.113:6789/0 conn(0x7fb300146ba0 legacy=0x7fb30006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.578+0000 7fb305a51640 1 -- 172.21.15.72:0/2879840762 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30006dad0 con 0x7fb30006cef0 2023-12-12T09:52:46.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.579+0000 7fb305a51640 1 --2- 172.21.15.72:0/2879840762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb30006cef0 0x7fb3000cce30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009e50 tx=0x7fb2f802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c16e90d37e35fff server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:46.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.579+0000 7fb2f7fff640 1 -- 172.21.15.72:0/2879840762 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f802cb00 con 0x7fb30006cef0 2023-12-12T09:52:46.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.579+0000 7fb2f7fff640 1 -- 172.21.15.72:0/2879840762 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f802cc60 con 0x7fb30006cef0 2023-12-12T09:52:46.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.579+0000 7fb2f7fff640 1 -- 172.21.15.72:0/2879840762 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80343f0 con 0x7fb30006cef0 2023-12-12T09:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.580+0000 7fb3074b4640 1 -- 172.21.15.72:0/2879840762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb30006cef0 msgr2=0x7fb3000cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.580+0000 7fb3074b4640 1 --2- 172.21.15.72:0/2879840762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb30006cef0 0x7fb3000cce30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009e50 tx=0x7fb2f802bf60 comp rx=0 tx=0).stop 2023-12-12T09:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.580+0000 7fb3074b4640 1 -- 172.21.15.72:0/2879840762 shutdown_connections 2023-12-12T09:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.580+0000 7fb3074b4640 1 --2- 172.21.15.72:0/2879840762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb30006cef0 0x7fb3000cce30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.580+0000 7fb3074b4640 1 --2- 172.21.15.72:0/2879840762 >> v2:172.21.15.113:3300/0 conn(0x7fb3001467c0 0x7fb3000cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.580+0000 7fb3074b4640 1 -- 172.21.15.72:0/2879840762 >> 172.21.15.72:0/2879840762 conn(0x7fb3000b0a90 msgr2=0x7fb3000b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.580+0000 7fb3074b4640 1 -- 172.21.15.72:0/2879840762 shutdown_connections 2023-12-12T09:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.580+0000 7fb3074b4640 1 -- 172.21.15.72:0/2879840762 wait complete. 2023-12-12T09:52:46.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.582+0000 7fb3074b4640 1 Processor -- start 2023-12-12T09:52:46.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.582+0000 7fb3074b4640 1 -- start start 2023-12-12T09:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb3074b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb30006cef0 0x7fb3000af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb3074b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3001467c0 0x7fb300137dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb3074b4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb300146ba0 0x7fb300138310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb3074b4640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001588e0 con 0x7fb300146ba0 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb3074b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001f7d00 con 0x7fb30006cef0 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb3074b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001f3650 con 0x7fb3001467c0 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb305a51640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb300146ba0 0x7fb300138310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb305a51640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb300146ba0 0x7fb300138310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59002/0 (socket says 172.21.15.72:59002) 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb305a51640 1 -- 172.21.15.72:0/208857185 learned_addr learned my addr 172.21.15.72:0/208857185 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb304a4f640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3001467c0 0x7fb300137dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.583+0000 7fb305250640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb30006cef0 0x7fb3000af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb305a51640 1 -- 172.21.15.72:0/208857185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3001467c0 msgr2=0x7fb300137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb305a51640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3001467c0 0x7fb300137dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb305a51640 1 -- 172.21.15.72:0/208857185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb30006cef0 msgr2=0x7fb3000af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb305a51640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb30006cef0 0x7fb3000af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb305a51640 1 -- 172.21.15.72:0/208857185 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f8009b30 con 0x7fb300146ba0 2023-12-12T09:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb305a51640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb300146ba0 0x7fb300138310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009e50 tx=0x7fb2f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb2f5ffb640 1 -- 172.21.15.72:0/208857185 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8009510 con 0x7fb300146ba0 2023-12-12T09:52:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb2f5ffb640 1 -- 172.21.15.72:0/208857185 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f803e450 con 0x7fb300146ba0 2023-12-12T09:52:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb2f5ffb640 1 -- 172.21.15.72:0/208857185 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f803e800 con 0x7fb300146ba0 2023-12-12T09:52:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb300157f40 con 0x7fb300146ba0 2023-12-12T09:52:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.584+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb300158500 con 0x7fb300146ba0 2023-12-12T09:52:46.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.585+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb300158a20 con 0x7fb300146ba0 2023-12-12T09:52:46.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.586+0000 7fb2f5ffb640 1 -- 172.21.15.72:0/208857185 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb2f803e960 con 0x7fb300146ba0 2023-12-12T09:52:46.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.587+0000 7fb2f5ffb640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb2e406ae70 0x7fb2e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:46.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.587+0000 7fb2f5ffb640 1 -- 172.21.15.72:0/208857185 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb2f80ac610 con 0x7fb300146ba0 2023-12-12T09:52:46.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.587+0000 7fb305250640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb2e406ae70 0x7fb2e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:46.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.588+0000 7fb305250640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb2e406ae70 0x7fb2e406d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009ad0 tx=0x7fb2fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:46.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.592+0000 7fb2f5ffb640 1 -- 172.21.15.72:0/208857185 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb300158a20 con 0x7fb300146ba0 2023-12-12T09:52:46.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.803+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb300158a20 con 0x7fb300146ba0 2023-12-12T09:52:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.804+0000 7fb2f5ffb640 1 -- 172.21.15.72:0/208857185 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fb300158a20 con 0x7fb300146ba0 2023-12-12T09:52:46.807 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:46.807 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:35.924557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:52:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-12-12T09:52:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.806+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb2e406ae70 msgr2=0x7fb2e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.806+0000 7fb3074b4640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb2e406ae70 0x7fb2e406d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009ad0 tx=0x7fb2fc008040 comp rx=0 tx=0).stop 2023-12-12T09:52:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.806+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb300146ba0 msgr2=0x7fb300138310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.806+0000 7fb3074b4640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb300146ba0 0x7fb300138310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009e50 tx=0x7fb2f8004a40 comp rx=0 tx=0).stop 2023-12-12T09:52:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.807+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 shutdown_connections 2023-12-12T09:52:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.807+0000 7fb3074b4640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb2e406ae70 0x7fb2e406d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.807+0000 7fb3074b4640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb300146ba0 0x7fb300138310 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.807+0000 7fb3074b4640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3001467c0 0x7fb300137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.807+0000 7fb3074b4640 1 --2- 172.21.15.72:0/208857185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb30006cef0 0x7fb3000af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.807+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 >> 172.21.15.72:0/208857185 conn(0x7fb3000b0a90 msgr2=0x7fb30015e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:46.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.807+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 shutdown_connections 2023-12-12T09:52:46.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:46.807+0000 7fb3074b4640 1 -- 172.21.15.72:0/208857185 wait complete. 2023-12-12T09:52:46.827 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.027+0000 7f4ce6c64640 1 Processor -- start 2023-12-12T09:52:47.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.027+0000 7f4ce6c64640 1 -- start start 2023-12-12T09:52:47.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.028+0000 7f4ce6c64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 0x7f4ce006c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.028+0000 7f4ce6c64640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4ce006de20 0x7f4ce015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.028+0000 7f4ce6c64640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce006ca40 con 0x7f4ce015e0e0 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.028+0000 7f4ce6c64640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce015e550 con 0x7f4ce006de20 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.028+0000 7f4ce6c64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0071cd0 con 0x7f4ce0157a60 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.028+0000 7f4cdffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4ce006de20 0x7f4ce015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.028+0000 7f4cdffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4ce006de20 0x7f4ce015dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59012/0 (socket says 172.21.15.72:59012) 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.028+0000 7f4cdffff640 1 -- 172.21.15.72:0/758448697 learned_addr learned my addr 172.21.15.72:0/758448697 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.029+0000 7f4ce4a00640 1 --2- 172.21.15.72:0/758448697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 0x7f4ce006c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.029+0000 7f4cdffff640 1 -- 172.21.15.72:0/758448697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 msgr2=0x7f4ce006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.029+0000 7f4cdffff640 1 --2- 172.21.15.72:0/758448697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 0x7f4ce006c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.029+0000 7f4cdffff640 1 -- 172.21.15.72:0/758448697 >> v1:172.21.15.72:6789/0 conn(0x7f4ce015e0e0 legacy=0x7f4ce015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.029+0000 7f4cdffff640 1 -- 172.21.15.72:0/758448697 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce0070f10 con 0x7f4ce006de20 2023-12-12T09:52:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.030+0000 7f4cdffff640 1 --2- 172.21.15.72:0/758448697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce006de20 0x7f4ce015dba0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009870 tx=0x7f4ccc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a261b5b6703d5c5c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.030+0000 7f4cdf7fe640 1 -- 172.21.15.72:0/758448697 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc031870 con 0x7f4ce006de20 2023-12-12T09:52:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.030+0000 7f4cdf7fe640 1 -- 172.21.15.72:0/758448697 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc0319d0 con 0x7f4ce006de20 2023-12-12T09:52:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.030+0000 7f4cdf7fe640 1 -- 172.21.15.72:0/758448697 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc031d60 con 0x7f4ce006de20 2023-12-12T09:52:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.031+0000 7f4ce6c64640 1 -- 172.21.15.72:0/758448697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce006de20 msgr2=0x7f4ce015dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.031+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/758448697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce006de20 0x7f4ce015dba0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009870 tx=0x7f4ccc02b910 comp rx=0 tx=0).stop 2023-12-12T09:52:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.031+0000 7f4ce6c64640 1 -- 172.21.15.72:0/758448697 shutdown_connections 2023-12-12T09:52:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.031+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/758448697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce006de20 0x7f4ce015dba0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.031+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/758448697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 0x7f4ce006c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.031+0000 7f4ce6c64640 1 -- 172.21.15.72:0/758448697 >> 172.21.15.72:0/758448697 conn(0x7f4ce00b0a90 msgr2=0x7f4ce00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.031+0000 7f4ce6c64640 1 -- 172.21.15.72:0/758448697 shutdown_connections 2023-12-12T09:52:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.031+0000 7f4ce6c64640 1 -- 172.21.15.72:0/758448697 wait complete. 2023-12-12T09:52:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.033+0000 7f4ce6c64640 1 Processor -- start 2023-12-12T09:52:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.033+0000 7f4ce6c64640 1 -- start start 2023-12-12T09:52:47.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce6c64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce006de20 0x7f4ce014b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce6c64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 0x7f4ce014ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce6c64640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce015e0e0 0x7f4ce0148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce6c64640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01496b0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce6c64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0148aa0 con 0x7f4ce006de20 2023-12-12T09:52:47.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce6c64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0148d80 con 0x7f4ce0157a60 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce5201640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce015e0e0 0x7f4ce0148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce5201640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce015e0e0 0x7f4ce0148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59016/0 (socket says 172.21.15.72:59016) 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce5201640 1 -- 172.21.15.72:0/3240627793 learned_addr learned my addr 172.21.15.72:0/3240627793 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4cdffff640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 0x7f4ce014ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce4a00640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce006de20 0x7f4ce014b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce5201640 1 -- 172.21.15.72:0/3240627793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 msgr2=0x7f4ce014ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce5201640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 0x7f4ce014ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce5201640 1 -- 172.21.15.72:0/3240627793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce006de20 msgr2=0x7f4ce014b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce5201640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce006de20 0x7f4ce014b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.034+0000 7f4ce5201640 1 -- 172.21.15.72:0/3240627793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc0094d0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.035+0000 7f4ce5201640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce015e0e0 0x7f4ce0148200 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800c6b0 tx=0x7f4cd800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.035+0000 7f4cdd7fa640 1 -- 172.21.15.72:0/3240627793 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8003b70 con 0x7f4ce015e0e0 2023-12-12T09:52:47.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.035+0000 7f4cdd7fa640 1 -- 172.21.15.72:0/3240627793 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd8003cf0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.035+0000 7f4cdd7fa640 1 -- 172.21.15.72:0/3240627793 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8003ea0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.035+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce0149060 con 0x7f4ce015e0e0 2023-12-12T09:52:47.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.035+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce014d3b0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.036+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce00b1e90 con 0x7f4ce015e0e0 2023-12-12T09:52:47.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.037+0000 7f4cdd7fa640 1 -- 172.21.15.72:0/3240627793 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4cd8005310 con 0x7f4ce015e0e0 2023-12-12T09:52:47.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.038+0000 7f4cdd7fa640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4cc406ae40 0x7f4cc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.038+0000 7f4cdd7fa640 1 -- 172.21.15.72:0/3240627793 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4cd80897b0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.038+0000 7f4ce4a00640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4cc406ae40 0x7f4cc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.039+0000 7f4ce4a00640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4cc406ae40 0x7f4cc406d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006fd0 tx=0x7f4cd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.042+0000 7f4cdd7fa640 1 -- 172.21.15.72:0/3240627793 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4cd80544d0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.231+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ce00745d0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.233+0000 7f4cdd7fa640 1 -- 172.21.15.72:0/3240627793 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f4cd8057af0 con 0x7f4ce015e0e0 2023-12-12T09:52:47.235 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:47.236 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4cc406ae40 msgr2=0x7f4cc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4cc406ae40 0x7f4cc406d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006fd0 tx=0x7f4cd4008040 comp rx=0 tx=0).stop 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce015e0e0 msgr2=0x7f4ce0148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce015e0e0 0x7f4ce0148200 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800c6b0 tx=0x7f4cd800cb70 comp rx=0 tx=0).stop 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 shutdown_connections 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4cc406ae40 0x7f4cc406d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce015e0e0 0x7f4ce0148200 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ce0157a60 0x7f4ce014ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 --2- 172.21.15.72:0/3240627793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ce006de20 0x7f4ce014b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 >> 172.21.15.72:0/3240627793 conn(0x7f4ce00b0a90 msgr2=0x7f4ce00cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 shutdown_connections 2023-12-12T09:52:47.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.236+0000 7f4ce6c64640 1 -- 172.21.15.72:0/3240627793 wait complete. 2023-12-12T09:52:47.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:52:47.258 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:52:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.473+0000 7fce7a11d640 1 Processor -- start 2023-12-12T09:52:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.473+0000 7fce7a11d640 1 -- start start 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce7a11d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce7415b1f0 0x7fce740d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce7a11d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fce740ceab0 0x7fce740cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce7a11d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7409a5a0 con 0x7fce7415b5d0 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce7a11d640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7406ebe0 con 0x7fce740ceab0 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce7a11d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7409a880 con 0x7fce7415b1f0 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce73fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fce740ceab0 0x7fce740cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce72ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fce7415b5d0 0x7fce7406e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59306/0 (socket says 172.21.15.72:59306) 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce73fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fce740ceab0 0x7fce740cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59026/0 (socket says 172.21.15.72:59026) 2023-12-12T09:52:47.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.475+0000 7fce72ffd640 1 -- 172.21.15.72:0/726601802 learned_addr learned my addr 172.21.15.72:0/726601802 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:47.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.476+0000 7fce73fff640 1 -- 172.21.15.72:0/726601802 >> v2:172.21.15.113:3300/0 conn(0x7fce7415b1f0 msgr2=0x7fce740d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:47.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.476+0000 7fce73fff640 1 --2- 172.21.15.72:0/726601802 >> v2:172.21.15.113:3300/0 conn(0x7fce7415b1f0 0x7fce740d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.476+0000 7fce73fff640 1 -- 172.21.15.72:0/726601802 >> v1:172.21.15.72:6789/0 conn(0x7fce7415b5d0 legacy=0x7fce7406e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.476+0000 7fce73fff640 1 -- 172.21.15.72:0/726601802 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740cbb20 con 0x7fce740ceab0 2023-12-12T09:52:47.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.477+0000 7fce73fff640 1 --2- 172.21.15.72:0/726601802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce740ceab0 0x7fce740cf170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce740b39e0 tx=0x7fce6c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d3a5dabf0e5ddd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.477+0000 7fce727fc640 1 -- 172.21.15.72:0/726601802 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c02cbe0 con 0x7fce740ceab0 2023-12-12T09:52:47.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.477+0000 7fce727fc640 1 -- 172.21.15.72:0/726601802 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c02cd40 con 0x7fce740ceab0 2023-12-12T09:52:47.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.477+0000 7fce727fc640 1 -- 172.21.15.72:0/726601802 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c034660 con 0x7fce740ceab0 2023-12-12T09:52:47.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.477+0000 7fce7a11d640 1 -- 172.21.15.72:0/726601802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce740ceab0 msgr2=0x7fce740cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.477+0000 7fce7a11d640 1 --2- 172.21.15.72:0/726601802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce740ceab0 0x7fce740cf170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce740b39e0 tx=0x7fce6c02be30 comp rx=0 tx=0).stop 2023-12-12T09:52:47.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.478+0000 7fce7a11d640 1 -- 172.21.15.72:0/726601802 shutdown_connections 2023-12-12T09:52:47.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.478+0000 7fce7a11d640 1 --2- 172.21.15.72:0/726601802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce740ceab0 0x7fce740cf170 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.478+0000 7fce7a11d640 1 --2- 172.21.15.72:0/726601802 >> v2:172.21.15.113:3300/0 conn(0x7fce7415b1f0 0x7fce740d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.478+0000 7fce7a11d640 1 -- 172.21.15.72:0/726601802 >> 172.21.15.72:0/726601802 conn(0x7fce740d0ee0 msgr2=0x7fce740d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:47.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.478+0000 7fce7a11d640 1 -- 172.21.15.72:0/726601802 shutdown_connections 2023-12-12T09:52:47.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.478+0000 7fce7a11d640 1 -- 172.21.15.72:0/726601802 wait complete. 2023-12-12T09:52:47.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.479+0000 7fce7a11d640 1 Processor -- start 2023-12-12T09:52:47.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.480+0000 7fce7a11d640 1 -- start start 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.480+0000 7fce7a11d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce740ceab0 0x7fce7414c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.480+0000 7fce7a11d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7415b1f0 0x7fce740ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.480+0000 7fce7a11d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce7415b5d0 0x7fce7415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce7a11d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7414da40 con 0x7fce7415b5d0 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce7a11d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7414cb50 con 0x7fce740ceab0 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce7a11d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7414ce30 con 0x7fce7415b1f0 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce73fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce7415b5d0 0x7fce7415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce73fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce7415b5d0 0x7fce7415dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59032/0 (socket says 172.21.15.72:59032) 2023-12-12T09:52:47.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce73fff640 1 -- 172.21.15.72:0/277513367 learned_addr learned my addr 172.21.15.72:0/277513367 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce73fff640 1 -- 172.21.15.72:0/277513367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7415b1f0 msgr2=0x7fce740ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce73fff640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7415b1f0 0x7fce740ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce73fff640 1 -- 172.21.15.72:0/277513367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce740ceab0 msgr2=0x7fce7414c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce73fff640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce740ceab0 0x7fce7414c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.481+0000 7fce73fff640 1 -- 172.21.15.72:0/277513367 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c009a70 con 0x7fce7415b5d0 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.482+0000 7fce73fff640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce7415b5d0 0x7fce7415dbe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce6c037040 tx=0x7fce6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.482+0000 7fce5bfff640 1 -- 172.21.15.72:0/277513367 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c03e610 con 0x7fce7415b5d0 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.482+0000 7fce5bfff640 1 -- 172.21.15.72:0/277513367 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c03e770 con 0x7fce7415b5d0 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.482+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7414d110 con 0x7fce7415b5d0 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.482+0000 7fce5bfff640 1 -- 172.21.15.72:0/277513367 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c03eae0 con 0x7fce7415b5d0 2023-12-12T09:52:47.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.482+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7414d6d0 con 0x7fce7415b5d0 2023-12-12T09:52:47.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.483+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce74074450 con 0x7fce7415b5d0 2023-12-12T09:52:47.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.484+0000 7fce5bfff640 1 -- 172.21.15.72:0/277513367 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce6c03b050 con 0x7fce7415b5d0 2023-12-12T09:52:47.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.485+0000 7fce5bfff640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce6006ae40 0x7fce6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.485+0000 7fce5bfff640 1 -- 172.21.15.72:0/277513367 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fce6c0ac4b0 con 0x7fce7415b5d0 2023-12-12T09:52:47.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.485+0000 7fce737fe640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce6006ae40 0x7fce6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.486+0000 7fce737fe640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce6006ae40 0x7fce6006d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fce64009710 tx=0x7fce64006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.490+0000 7fce5bfff640 1 -- 172.21.15.72:0/277513367 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce6c0771d0 con 0x7fce7415b5d0 2023-12-12T09:52:47.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.682+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce740b6e60 con 0x7fce7415b5d0 2023-12-12T09:52:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.682+0000 7fce5bfff640 1 -- 172.21.15.72:0/277513367 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce6c07a7f0 con 0x7fce7415b5d0 2023-12-12T09:52:47.685 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:52:47.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce6006ae40 msgr2=0x7fce6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce6006ae40 0x7fce6006d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fce64009710 tx=0x7fce64006c90 comp rx=0 tx=0).stop 2023-12-12T09:52:47.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce7415b5d0 msgr2=0x7fce7415dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce7415b5d0 0x7fce7415dbe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce6c037040 tx=0x7fce6c004a40 comp rx=0 tx=0).stop 2023-12-12T09:52:47.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 shutdown_connections 2023-12-12T09:52:47.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce6006ae40 0x7fce6006d300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce7415b5d0 0x7fce7415dbe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7415b1f0 0x7fce740ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 --2- 172.21.15.72:0/277513367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce740ceab0 0x7fce7414c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 >> 172.21.15.72:0/277513367 conn(0x7fce740d0ee0 msgr2=0x7fce740b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:47.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 shutdown_connections 2023-12-12T09:52:47.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.685+0000 7fce7a11d640 1 -- 172.21.15.72:0/277513367 wait complete. 2023-12-12T09:52:47.706 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:52:47.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.922+0000 7fd19b59e640 1 Processor -- start 2023-12-12T09:52:47.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.922+0000 7fd19b59e640 1 -- start start 2023-12-12T09:52:47.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.923+0000 7fd19b59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 0x7fd19c0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.924+0000 7fd19b59e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0cc310 con 0x7fd19c143a40 2023-12-12T09:52:47.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.924+0000 7fd19b59e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0cc5f0 con 0x7fd19c143660 2023-12-12T09:52:47.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.924+0000 7fd19b59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c070180 con 0x7fd19c0ce2b0 2023-12-12T09:52:47.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.924+0000 7fd193fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd19c143a40 0x7fd19c0cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59310/0 (socket says 172.21.15.72:59310) 2023-12-12T09:52:47.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.924+0000 7fd193fff640 1 -- 172.21.15.72:0/83520340 learned_addr learned my addr 172.21.15.72:0/83520340 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:47.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.924+0000 7fd19ad9d640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 0x7fd19c0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.924+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920745151 0 0) 0x7fd19c0cc310 con 0x7fd19c143a40 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd188003520 con 0x7fd19c143a40 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2623952879 0 0) 0x7fd188003520 con 0x7fd19c143a40 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 msgr2=0x7fd19c0cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 0x7fd19c0cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 >> v1:172.21.15.113:6789/0 conn(0x7fd19c143660 legacy=0x7fd19c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c070460 con 0x7fd19c143a40 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fd184001670 con 0x7fd19c143a40 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 >> v1:172.21.15.72:6789/0 conn(0x7fd19c143a40 legacy=0x7fd19c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd188003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd188004350 0x7fd188004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 0x7fd188005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c070180 con 0x7fd1880039a0 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18800b9b0 con 0x7fd188004350 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18800bb70 con 0x7fd19c0ce2b0 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.925+0000 7fd19a59c640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd188003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.926+0000 7fd193fff640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd188004350 0x7fd188004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.926+0000 7fd19ad9d640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 0x7fd188005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.926+0000 7fd19a59c640 1 -- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 msgr2=0x7fd188005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:47.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.926+0000 7fd19a59c640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 0x7fd188005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.926+0000 7fd19a59c640 1 -- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd188004350 msgr2=0x7fd188004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:47.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.926+0000 7fd19a59c640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd188004350 0x7fd188004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.926+0000 7fd19a59c640 1 -- 172.21.15.72:0/83520340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188003520 con 0x7fd1880039a0 2023-12-12T09:52:47.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.926+0000 7fd19ad9d640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c0ce2b0 0x7fd188005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:47.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.927+0000 7fd19a59c640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd188003d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001910 tx=0x7fd1880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6deb6de14009b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.927+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c032650 con 0x7fd1880039a0 2023-12-12T09:52:47.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.927+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c0327b0 con 0x7fd1880039a0 2023-12-12T09:52:47.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.927+0000 7fd199d9b640 1 -- 172.21.15.72:0/83520340 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c032b40 con 0x7fd1880039a0 2023-12-12T09:52:47.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.928+0000 7fd19b59e640 1 -- 172.21.15.72:0/83520340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 msgr2=0x7fd188003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.928+0000 7fd19b59e640 1 --2- 172.21.15.72:0/83520340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd188003d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001910 tx=0x7fd1880057f0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.928+0000 7fd19ad9d640 1 -- 172.21.15.72:0/83520340 reap_dead start 2023-12-12T09:52:47.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.928+0000 7fd19b59e640 1 -- 172.21.15.72:0/83520340 shutdown_connections 2023-12-12T09:52:47.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.928+0000 7fd19b59e640 1 -- 172.21.15.72:0/83520340 >> 172.21.15.72:0/83520340 conn(0x7fd19c0b0940 msgr2=0x7fd19c0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:47.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.928+0000 7fd19b59e640 1 -- 172.21.15.72:0/83520340 shutdown_connections 2023-12-12T09:52:47.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.928+0000 7fd19b59e640 1 -- 172.21.15.72:0/83520340 wait complete. 2023-12-12T09:52:47.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.930+0000 7fd19b59e640 1 Processor -- start 2023-12-12T09:52:47.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.930+0000 7fd19b59e640 1 -- start start 2023-12-12T09:52:47.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19b59e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd19c09c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19b59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c137950 0x7fd19c13c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c0cca90 0x7fd19c0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19b59e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0cd450 con 0x7fd1880039a0 2023-12-12T09:52:47.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19b59e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c138cd0 con 0x7fd19c0cca90 2023-12-12T09:52:47.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19b59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c13c730 con 0x7fd19c137950 2023-12-12T09:52:47.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19a59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd19c09c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19a59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd19c09c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59052/0 (socket says 172.21.15.72:59052) 2023-12-12T09:52:47.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.931+0000 7fd19a59c640 1 -- 172.21.15.72:0/3463722919 learned_addr learned my addr 172.21.15.72:0/3463722919 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:47.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd193fff640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c137950 0x7fd19c13c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19ad9d640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c0cca90 0x7fd19c0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19a59c640 1 -- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c137950 msgr2=0x7fd19c13c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19a59c640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c137950 0x7fd19c13c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19a59c640 1 -- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c0cca90 msgr2=0x7fd19c0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:47.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19a59c640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c0cca90 0x7fd19c0ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:47.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19a59c640 1 -- 172.21.15.72:0/3463722919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c1380c0 con 0x7fd1880039a0 2023-12-12T09:52:47.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19ad9d640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c0cca90 0x7fd19c0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:47.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19a59c640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd19c09c000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001910 tx=0x7fd18c0012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd192ffd640 1 -- 172.21.15.72:0/3463722919 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0068d0 con 0x7fd1880039a0 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd192ffd640 1 -- 172.21.15.72:0/3463722919 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c006a30 con 0x7fd1880039a0 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c1383a0 con 0x7fd1880039a0 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.932+0000 7fd192ffd640 1 -- 172.21.15.72:0/3463722919 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c006da0 con 0x7fd1880039a0 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.933+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c138960 con 0x7fd1880039a0 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.934+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c067910 con 0x7fd1880039a0 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.935+0000 7fd192ffd640 1 -- 172.21.15.72:0/3463722919 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd18c042070 con 0x7fd1880039a0 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.936+0000 7fd192ffd640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd17c06ae40 0x7fd17c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:47.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.936+0000 7fd192ffd640 1 -- 172.21.15.72:0/3463722919 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd18c0abc30 con 0x7fd1880039a0 2023-12-12T09:52:47.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.936+0000 7fd193fff640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd17c06ae40 0x7fd17c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:47.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.937+0000 7fd193fff640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd17c06ae40 0x7fd17c06d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd184001050 tx=0x7fd184009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:47.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:47.941+0000 7fd192ffd640 1 -- 172.21.15.72:0/3463722919 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd18c076a70 con 0x7fd1880039a0 2023-12-12T09:52:48.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.137+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd19c066d20 con 0x7fd1880039a0 2023-12-12T09:52:48.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.137+0000 7fd192ffd640 1 -- 172.21.15.72:0/3463722919 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd18c07a090 con 0x7fd1880039a0 2023-12-12T09:52:48.140 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd17c06ae40 msgr2=0x7fd17c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd17c06ae40 0x7fd17c06d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd184001050 tx=0x7fd184009040 comp rx=0 tx=0).stop 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 msgr2=0x7fd19c09c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd19c09c000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001910 tx=0x7fd18c0012b0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 shutdown_connections 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd17c06ae40 0x7fd17c06d300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c0cca90 0x7fd19c0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c137950 0x7fd19c13c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 --2- 172.21.15.72:0/3463722919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1880039a0 0x7fd19c09c000 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 >> 172.21.15.72:0/3463722919 conn(0x7fd19c0b0940 msgr2=0x7fd19c0cb620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:48.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 shutdown_connections 2023-12-12T09:52:48.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.140+0000 7fd19b59e640 1 -- 172.21.15.72:0/3463722919 wait complete. 2023-12-12T09:52:48.161 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:52:48.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.366+0000 7f593b53c640 1 Processor -- start 2023-12-12T09:52:48.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.366+0000 7f593b53c640 1 -- start start 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f593b53c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f59340ce150 0x7f59340ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f593b53c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340cedf0 0x7f59340cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f593b53c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59340cf790 con 0x7f5934155400 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f593b53c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593406c500 con 0x7f59340ce150 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f593b53c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593415b4a0 con 0x7f59340cedf0 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f5939d39640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f59340ce150 0x7f59340ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f5939d39640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f59340ce150 0x7f59340ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59062/0 (socket says 172.21.15.72:59062) 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f5939d39640 1 -- 172.21.15.72:0/796214906 learned_addr learned my addr 172.21.15.72:0/796214906 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f5939d39640 1 -- 172.21.15.72:0/796214906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340cedf0 msgr2=0x7f59340cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f593ad3b640 1 --2- 172.21.15.72:0/796214906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340cedf0 0x7f59340cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f5939d39640 1 --2- 172.21.15.72:0/796214906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340cedf0 0x7f59340cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f5939d39640 1 -- 172.21.15.72:0/796214906 >> v1:172.21.15.113:6789/0 conn(0x7f5934155400 legacy=0x7f593406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:48.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.368+0000 7f5939d39640 1 -- 172.21.15.72:0/796214906 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340adbb0 con 0x7f59340ce150 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.369+0000 7f5939d39640 1 --2- 172.21.15.72:0/796214906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340ce150 0x7f59340ce5b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5934070020 tx=0x7f592002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2389a27933ed89ba server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.370+0000 7f5939538640 1 -- 172.21.15.72:0/796214906 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920031640 con 0x7f59340ce150 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.370+0000 7f5939538640 1 -- 172.21.15.72:0/796214906 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200317a0 con 0x7f59340ce150 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.370+0000 7f5939538640 1 -- 172.21.15.72:0/796214906 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920031b30 con 0x7f59340ce150 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.370+0000 7f593b53c640 1 -- 172.21.15.72:0/796214906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340ce150 msgr2=0x7f59340ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.370+0000 7f593b53c640 1 --2- 172.21.15.72:0/796214906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340ce150 0x7f59340ce5b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5934070020 tx=0x7f592002b8f0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.371+0000 7f593b53c640 1 -- 172.21.15.72:0/796214906 shutdown_connections 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.371+0000 7f593b53c640 1 --2- 172.21.15.72:0/796214906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340cedf0 0x7f59340cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.371+0000 7f593b53c640 1 --2- 172.21.15.72:0/796214906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340ce150 0x7f59340ce5b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.371+0000 7f593b53c640 1 -- 172.21.15.72:0/796214906 >> 172.21.15.72:0/796214906 conn(0x7f593406a640 msgr2=0x7f593406aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.371+0000 7f593b53c640 1 -- 172.21.15.72:0/796214906 shutdown_connections 2023-12-12T09:52:48.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.371+0000 7f593b53c640 1 -- 172.21.15.72:0/796214906 wait complete. 2023-12-12T09:52:48.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.372+0000 7f593b53c640 1 Processor -- start 2023-12-12T09:52:48.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.373+0000 7f593b53c640 1 -- start start 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f593b53c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340ce150 0x7f5934140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f593b53c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340cedf0 0x7f59340afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f593b53c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5934155400 0x7f5934152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f593b53c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59340af9c0 con 0x7f59340cedf0 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f593b53c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5934140b90 con 0x7f59340ce150 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f593b53c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59340b0320 con 0x7f5934155400 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f5939d39640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340cedf0 0x7f59340afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f5939d39640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340cedf0 0x7f59340afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59070/0 (socket says 172.21.15.72:59070) 2023-12-12T09:52:48.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f5939d39640 1 -- 172.21.15.72:0/2305124949 learned_addr learned my addr 172.21.15.72:0/2305124949 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:48.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f593a53a640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340ce150 0x7f5934140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f5939d39640 1 -- 172.21.15.72:0/2305124949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5934155400 msgr2=0x7f5934152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f5939d39640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5934155400 0x7f5934152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f5939d39640 1 -- 172.21.15.72:0/2305124949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340ce150 msgr2=0x7f5934140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f5939d39640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340ce150 0x7f5934140650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.374+0000 7f5939d39640 1 -- 172.21.15.72:0/2305124949 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59200094d0 con 0x7f59340cedf0 2023-12-12T09:52:48.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.375+0000 7f5939d39640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340cedf0 0x7f59340afde0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5920000c00 tx=0x7f59200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:48.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.375+0000 7f5932ffd640 1 -- 172.21.15.72:0/2305124949 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592003ac60 con 0x7f59340cedf0 2023-12-12T09:52:48.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.375+0000 7f5932ffd640 1 -- 172.21.15.72:0/2305124949 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592003ade0 con 0x7f59340cedf0 2023-12-12T09:52:48.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.375+0000 7f5932ffd640 1 -- 172.21.15.72:0/2305124949 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920044630 con 0x7f59340cedf0 2023-12-12T09:52:48.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.375+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59340aed40 con 0x7f59340cedf0 2023-12-12T09:52:48.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.375+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59340af300 con 0x7f59340cedf0 2023-12-12T09:52:48.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.376+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5934074450 con 0x7f59340cedf0 2023-12-12T09:52:48.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.377+0000 7f5932ffd640 1 -- 172.21.15.72:0/2305124949 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f592002f050 con 0x7f59340cedf0 2023-12-12T09:52:48.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.378+0000 7f5932ffd640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f591806ae40 0x7f591806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.378+0000 7f5932ffd640 1 -- 172.21.15.72:0/2305124949 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59200aba60 con 0x7f59340cedf0 2023-12-12T09:52:48.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.378+0000 7f593a53a640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f591806ae40 0x7f591806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.379+0000 7f593a53a640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f591806ae40 0x7f591806d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5928007dc0 tx=0x7f59280072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:48.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.382+0000 7f5932ffd640 1 -- 172.21.15.72:0/2305124949 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5920076820 con 0x7f59340cedf0 2023-12-12T09:52:48.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.577+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f593406d750 con 0x7f59340cedf0 2023-12-12T09:52:48.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.578+0000 7f5932ffd640 1 -- 172.21.15.72:0/2305124949 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5920032040 con 0x7f59340cedf0 2023-12-12T09:52:48.580 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:52:48.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.580+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f591806ae40 msgr2=0x7f591806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.580+0000 7f593b53c640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f591806ae40 0x7f591806d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5928007dc0 tx=0x7f59280072f0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.580+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340cedf0 msgr2=0x7f59340afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.580+0000 7f593b53c640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340cedf0 0x7f59340afde0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5920000c00 tx=0x7f59200046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.580+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 shutdown_connections 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.581+0000 7f593b53c640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f591806ae40 0x7f591806d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.581+0000 7f593b53c640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5934155400 0x7f5934152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.581+0000 7f593b53c640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59340cedf0 0x7f59340afde0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.581+0000 7f593b53c640 1 --2- 172.21.15.72:0/2305124949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340ce150 0x7f5934140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.581+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 >> 172.21.15.72:0/2305124949 conn(0x7f593406a640 msgr2=0x7f59340b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.581+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 shutdown_connections 2023-12-12T09:52:48.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.581+0000 7f593b53c640 1 -- 172.21.15.72:0/2305124949 wait complete. 2023-12-12T09:52:48.602 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:52:48.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.819+0000 7f040294c640 1 Processor -- start 2023-12-12T09:52:48.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.819+0000 7f040294c640 1 -- start start 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f040294c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc140290 0x7f03fc06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f040294c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f03fc071190 0x7f03fc140d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f040294c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc1415b0 con 0x7f03fc070d50 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f040294c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03fc06a5e0 con 0x7f03fc071190 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f040294c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc06a8c0 con 0x7f03fc140290 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f0400ee9640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f03fc071190 0x7f03fc140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f0400ee9640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f03fc071190 0x7f03fc140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59072/0 (socket says 172.21.15.72:59072) 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f0400ee9640 1 -- 172.21.15.72:0/1872858216 learned_addr learned my addr 172.21.15.72:0/1872858216 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.821+0000 7f03fbfff640 1 --2- 172.21.15.72:0/1872858216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc140290 0x7f03fc06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.822+0000 7f0400ee9640 1 -- 172.21.15.72:0/1872858216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc140290 msgr2=0x7f03fc06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:48.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.822+0000 7f0400ee9640 1 --2- 172.21.15.72:0/1872858216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc140290 0x7f03fc06b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.822+0000 7f0400ee9640 1 -- 172.21.15.72:0/1872858216 >> v1:172.21.15.113:6789/0 conn(0x7f03fc070d50 legacy=0x7f03fc140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:48.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.822+0000 7f0400ee9640 1 -- 172.21.15.72:0/1872858216 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc06aba0 con 0x7f03fc071190 2023-12-12T09:52:48.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.823+0000 7f0400ee9640 1 --2- 172.21.15.72:0/1872858216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc071190 0x7f03fc140d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0661f0 tx=0x7f03f402be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c987830c9943026 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:48.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.823+0000 7f03faffd640 1 -- 172.21.15.72:0/1872858216 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f402c9a0 con 0x7f03fc071190 2023-12-12T09:52:48.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.823+0000 7f03faffd640 1 -- 172.21.15.72:0/1872858216 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f402cb00 con 0x7f03fc071190 2023-12-12T09:52:48.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.823+0000 7f03faffd640 1 -- 172.21.15.72:0/1872858216 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f402ce90 con 0x7f03fc071190 2023-12-12T09:52:48.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.824+0000 7f040294c640 1 -- 172.21.15.72:0/1872858216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc071190 msgr2=0x7f03fc140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.824+0000 7f040294c640 1 --2- 172.21.15.72:0/1872858216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc071190 0x7f03fc140d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0661f0 tx=0x7f03f402be00 comp rx=0 tx=0).stop 2023-12-12T09:52:48.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.824+0000 7f040294c640 1 -- 172.21.15.72:0/1872858216 shutdown_connections 2023-12-12T09:52:48.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.824+0000 7f040294c640 1 --2- 172.21.15.72:0/1872858216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc071190 0x7f03fc140d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.824+0000 7f040294c640 1 --2- 172.21.15.72:0/1872858216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc140290 0x7f03fc06b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.824+0000 7f040294c640 1 -- 172.21.15.72:0/1872858216 >> 172.21.15.72:0/1872858216 conn(0x7f03fc0b1540 msgr2=0x7f03fc0b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:48.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.824+0000 7f040294c640 1 -- 172.21.15.72:0/1872858216 shutdown_connections 2023-12-12T09:52:48.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.824+0000 7f040294c640 1 -- 172.21.15.72:0/1872858216 wait complete. 2023-12-12T09:52:48.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.826+0000 7f040294c640 1 Processor -- start 2023-12-12T09:52:48.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.826+0000 7f040294c640 1 -- start start 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f040294c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03fc070d50 0x7f03fc15c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f040294c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc071190 0x7f03fc13f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f040294c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc140290 0x7f03fc1f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f040294c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc15d9b0 con 0x7f03fc140290 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f040294c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1581e0 con 0x7f03fc070d50 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f040294c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc15cda0 con 0x7f03fc071190 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f0400ee9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc140290 0x7f03fc1f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f0400ee9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc140290 0x7f03fc1f1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59074/0 (socket says 172.21.15.72:59074) 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f0400ee9640 1 -- 172.21.15.72:0/2513075371 learned_addr learned my addr 172.21.15.72:0/2513075371 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f03fbfff640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03fc070d50 0x7f03fc15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f0400ee9640 1 -- 172.21.15.72:0/2513075371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc071190 msgr2=0x7f03fc13f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f0400ee9640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc071190 0x7f03fc13f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f0400ee9640 1 -- 172.21.15.72:0/2513075371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03fc070d50 msgr2=0x7f03fc15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:48.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f0400ee9640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03fc070d50 0x7f03fc15c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.828+0000 7f0400ee9640 1 -- 172.21.15.72:0/2513075371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009b30 con 0x7f03fc140290 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.829+0000 7f0400ee9640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc140290 0x7f03fc1f1330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009f40 tx=0x7f03f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.829+0000 7f03f8ff9640 1 -- 172.21.15.72:0/2513075371 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4009510 con 0x7f03fc140290 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.829+0000 7f03f8ff9640 1 -- 172.21.15.72:0/2513075371 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f403e450 con 0x7f03fc140290 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.829+0000 7f03f8ff9640 1 -- 172.21.15.72:0/2513075371 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f403e7e0 con 0x7f03fc140290 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.829+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc15d080 con 0x7f03fc140290 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.829+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc15d640 con 0x7f03fc140290 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.830+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc15db60 con 0x7f03fc140290 2023-12-12T09:52:48.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.831+0000 7f03f8ff9640 1 -- 172.21.15.72:0/2513075371 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f03f403e940 con 0x7f03fc140290 2023-12-12T09:52:48.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.832+0000 7f03f8ff9640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03e806ae40 0x7f03e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:48.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.832+0000 7f03f8ff9640 1 -- 172.21.15.72:0/2513075371 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f03f40acab0 con 0x7f03fc140290 2023-12-12T09:52:48.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.832+0000 7f03fbfff640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03e806ae40 0x7f03e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:48.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.833+0000 7f03fbfff640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03e806ae40 0x7f03e806d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03ec006fd0 tx=0x7f03ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:48.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:48.836+0000 7f03f8ff9640 1 -- 172.21.15.72:0/2513075371 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03fc15db60 con 0x7f03fc140290 2023-12-12T09:52:49.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.026+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f03fc15db60 con 0x7f03fc140290 2023-12-12T09:52:49.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.027+0000 7f03f8ff9640 1 -- 172.21.15.72:0/2513075371 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f03f4043020 con 0x7f03fc140290 2023-12-12T09:52:49.029 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:52:49.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.029+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03e806ae40 msgr2=0x7f03e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.029+0000 7f040294c640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03e806ae40 0x7f03e806d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03ec006fd0 tx=0x7f03ec008040 comp rx=0 tx=0).stop 2023-12-12T09:52:49.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.029+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc140290 msgr2=0x7f03fc1f1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.029+0000 7f040294c640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc140290 0x7f03fc1f1330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009f40 tx=0x7f03f4004a40 comp rx=0 tx=0).stop 2023-12-12T09:52:49.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.029+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 shutdown_connections 2023-12-12T09:52:49.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.029+0000 7f040294c640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f03e806ae40 0x7f03e806d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.029+0000 7f040294c640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03fc140290 0x7f03fc1f1330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.030+0000 7f040294c640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03fc071190 0x7f03fc13f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.030+0000 7f040294c640 1 --2- 172.21.15.72:0/2513075371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03fc070d50 0x7f03fc15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.030+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 >> 172.21.15.72:0/2513075371 conn(0x7f03fc0b1540 msgr2=0x7f03fc15f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:49.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.030+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 shutdown_connections 2023-12-12T09:52:49.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.030+0000 7f040294c640 1 -- 172.21.15.72:0/2513075371 wait complete. 2023-12-12T09:52:49.052 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:52:49.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.255+0000 7f4172cf3640 1 Processor -- start 2023-12-12T09:52:49.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.255+0000 7f4172cf3640 1 -- start start 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.256+0000 7f4172cf3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f416c143e00 0x7f416c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.256+0000 7f4172cf3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c06c410 0x7f416c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.256+0000 7f4172cf3640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416c0b3bb0 con 0x7f416c072180 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.256+0000 7f4172cf3640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c1441e0 con 0x7f416c143e00 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.256+0000 7f4172cf3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c072680 con 0x7f416c06c410 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f4170a8f640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f416c143e00 0x7f416c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f416bfff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f416c072180 0x7f416c0b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59326/0 (socket says 172.21.15.72:59326) 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f416bfff640 1 -- 172.21.15.72:0/2056179092 learned_addr learned my addr 172.21.15.72:0/2056179092 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f4171290640 1 --2- 172.21.15.72:0/2056179092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c06c410 0x7f416c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f4170a8f640 1 -- 172.21.15.72:0/2056179092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c06c410 msgr2=0x7f416c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f4170a8f640 1 --2- 172.21.15.72:0/2056179092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c06c410 0x7f416c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f4170a8f640 1 -- 172.21.15.72:0/2056179092 >> v1:172.21.15.72:6789/0 conn(0x7f416c072180 legacy=0x7f416c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f4170a8f640 1 -- 172.21.15.72:0/2056179092 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c0b1e30 con 0x7f416c143e00 2023-12-12T09:52:49.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.257+0000 7f416b7fe640 1 -- 172.21.15.72:0/2056179092 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524589161 0 0) 0x7f416c0b3bb0 con 0x7f416c072180 2023-12-12T09:52:49.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.258+0000 7f4170a8f640 1 --2- 172.21.15.72:0/2056179092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c143e00 0x7f416c0b4040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41600097a0 tx=0x7f416002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d992b352f6fadf server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:49.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.258+0000 7f416b7fe640 1 -- 172.21.15.72:0/2056179092 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160031830 con 0x7f416c143e00 2023-12-12T09:52:49.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f416b7fe640 1 -- 172.21.15.72:0/2056179092 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160031990 con 0x7f416c143e00 2023-12-12T09:52:49.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f416b7fe640 1 -- 172.21.15.72:0/2056179092 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160031d20 con 0x7f416c143e00 2023-12-12T09:52:49.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f4172cf3640 1 -- 172.21.15.72:0/2056179092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c143e00 msgr2=0x7f416c0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f4172cf3640 1 --2- 172.21.15.72:0/2056179092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c143e00 0x7f416c0b4040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41600097a0 tx=0x7f416002b8d0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f4172cf3640 1 -- 172.21.15.72:0/2056179092 shutdown_connections 2023-12-12T09:52:49.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f4172cf3640 1 --2- 172.21.15.72:0/2056179092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c06c410 0x7f416c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f4172cf3640 1 --2- 172.21.15.72:0/2056179092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c143e00 0x7f416c0b4040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f4172cf3640 1 -- 172.21.15.72:0/2056179092 >> 172.21.15.72:0/2056179092 conn(0x7f416c0b0a90 msgr2=0x7f416c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:49.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f4172cf3640 1 -- 172.21.15.72:0/2056179092 shutdown_connections 2023-12-12T09:52:49.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.259+0000 7f4172cf3640 1 -- 172.21.15.72:0/2056179092 wait complete. 2023-12-12T09:52:49.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.261+0000 7f4172cf3640 1 Processor -- start 2023-12-12T09:52:49.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.261+0000 7f4172cf3640 1 -- start start 2023-12-12T09:52:49.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.262+0000 7f4172cf3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c06c410 0x7f416c13abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.262+0000 7f4172cf3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c072180 0x7f416c14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.262+0000 7f4172cf3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f416c143e00 0x7f416c14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.262+0000 7f4172cf3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c140cf0 con 0x7f416c06c410 2023-12-12T09:52:49.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.262+0000 7f4172cf3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1f28f0 con 0x7f416c143e00 2023-12-12T09:52:49.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.262+0000 7f4172cf3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c13d5c0 con 0x7f416c072180 2023-12-12T09:52:49.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c06c410 0x7f416c13abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c06c410 0x7f416c13abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59100/0 (socket says 172.21.15.72:59100) 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 -- 172.21.15.72:0/1600514522 learned_addr learned my addr 172.21.15.72:0/1600514522 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f416bfff640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c072180 0x7f416c14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 -- 172.21.15.72:0/1600514522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c072180 msgr2=0x7f416c14e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c072180 0x7f416c14e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 -- 172.21.15.72:0/1600514522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f416c143e00 msgr2=0x7f416c14ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f416c143e00 0x7f416c14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 -- 172.21.15.72:0/1600514522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4160009400 con 0x7f416c06c410 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f4170a8f640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c06c410 0x7f416c13abd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f41600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.263+0000 7f41697fa640 1 -- 172.21.15.72:0/1600514522 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416003aa60 con 0x7f416c06c410 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.264+0000 7f41697fa640 1 -- 172.21.15.72:0/1600514522 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416003abc0 con 0x7f416c06c410 2023-12-12T09:52:49.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.264+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c13d8a0 con 0x7f416c06c410 2023-12-12T09:52:49.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.264+0000 7f41697fa640 1 -- 172.21.15.72:0/1600514522 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41600443f0 con 0x7f416c06c410 2023-12-12T09:52:49.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.264+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c14f2a0 con 0x7f416c06c410 2023-12-12T09:52:49.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.265+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c140e60 con 0x7f416c06c410 2023-12-12T09:52:49.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.266+0000 7f41697fa640 1 -- 172.21.15.72:0/1600514522 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f416004d450 con 0x7f416c06c410 2023-12-12T09:52:49.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.267+0000 7f41697fa640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f415006ae40 0x7f415006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.267+0000 7f41697fa640 1 -- 172.21.15.72:0/1600514522 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f41600abad0 con 0x7f416c06c410 2023-12-12T09:52:49.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.267+0000 7f416bfff640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f415006ae40 0x7f415006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.268+0000 7f416bfff640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f415006ae40 0x7f415006d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4158007fd0 tx=0x7f4158009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:49.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.271+0000 7f41697fa640 1 -- 172.21.15.72:0/1600514522 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41600767f0 con 0x7f416c06c410 2023-12-12T09:52:49.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.496+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f416c0745d0 con 0x7f416c06c410 2023-12-12T09:52:49.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.497+0000 7f41697fa640 1 -- 172.21.15.72:0/1600514522 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f4160035020 con 0x7f416c06c410 2023-12-12T09:52:49.499 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:49.499 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:35.924557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:52:49.499 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-12-12T09:52:49.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.499+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f415006ae40 msgr2=0x7f415006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.499+0000 7f4172cf3640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f415006ae40 0x7f415006d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4158007fd0 tx=0x7f4158009040 comp rx=0 tx=0).stop 2023-12-12T09:52:49.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.499+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c06c410 msgr2=0x7f416c13abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c06c410 0x7f416c13abd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f41600046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 shutdown_connections 2023-12-12T09:52:49.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f415006ae40 0x7f415006d300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f416c143e00 0x7f416c14ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f416c072180 0x7f416c14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 --2- 172.21.15.72:0/1600514522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c06c410 0x7f416c13abd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 >> 172.21.15.72:0/1600514522 conn(0x7f416c0b0a90 msgr2=0x7f416c0af630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:49.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 shutdown_connections 2023-12-12T09:52:49.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.500+0000 7f4172cf3640 1 -- 172.21.15.72:0/1600514522 wait complete. 2023-12-12T09:52:49.519 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:52:49.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.731+0000 7fd29ab0f640 1 Processor -- start 2023-12-12T09:52:49.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.731+0000 7fd29ab0f640 1 -- start start 2023-12-12T09:52:49.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd29ab0f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd294143e00 0x7fd2940b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd29ab0f640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd294072180 0x7fd2940b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd29ab0f640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940b3bb0 con 0x7fd29406c410 2023-12-12T09:52:49.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd29ab0f640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2941441e0 con 0x7fd294072180 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd29ab0f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd294072680 con 0x7fd294143e00 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd293fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd294072180 0x7fd2940b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd2990ac640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd29406c410 0x7fd294070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59338/0 (socket says 172.21.15.72:59338) 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd293fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd294072180 0x7fd2940b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59102/0 (socket says 172.21.15.72:59102) 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd2990ac640 1 -- 172.21.15.72:0/2211409118 learned_addr learned my addr 172.21.15.72:0/2211409118 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd2988ab640 1 --2- 172.21.15.72:0/2211409118 >> v2:172.21.15.113:3300/0 conn(0x7fd294143e00 0x7fd2940b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.733+0000 7fd293fff640 1 -- 172.21.15.72:0/2211409118 >> v2:172.21.15.113:3300/0 conn(0x7fd294143e00 msgr2=0x7fd2940b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.734+0000 7fd293fff640 1 --2- 172.21.15.72:0/2211409118 >> v2:172.21.15.113:3300/0 conn(0x7fd294143e00 0x7fd2940b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.734+0000 7fd293fff640 1 -- 172.21.15.72:0/2211409118 >> v1:172.21.15.72:6789/0 conn(0x7fd29406c410 legacy=0x7fd294070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.734+0000 7fd293fff640 1 -- 172.21.15.72:0/2211409118 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940b1e30 con 0x7fd294072180 2023-12-12T09:52:49.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.734+0000 7fd2988ab640 1 --2- 172.21.15.72:0/2211409118 >> v2:172.21.15.113:3300/0 conn(0x7fd294143e00 0x7fd2940b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:49.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.735+0000 7fd293fff640 1 --2- 172.21.15.72:0/2211409118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd2940b4580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd29406b760 tx=0x7fd28002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f01093a5e0febc server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:49.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.735+0000 7fd2937fe640 1 -- 172.21.15.72:0/2211409118 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280031830 con 0x7fd294072180 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.735+0000 7fd2937fe640 1 -- 172.21.15.72:0/2211409118 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280031990 con 0x7fd294072180 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.735+0000 7fd2937fe640 1 -- 172.21.15.72:0/2211409118 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280031d20 con 0x7fd294072180 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.736+0000 7fd29ab0f640 1 -- 172.21.15.72:0/2211409118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 msgr2=0x7fd2940b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.736+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/2211409118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd2940b4580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd29406b760 tx=0x7fd28002b8d0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.736+0000 7fd29ab0f640 1 -- 172.21.15.72:0/2211409118 shutdown_connections 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.736+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/2211409118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd2940b4580 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.736+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/2211409118 >> v2:172.21.15.113:3300/0 conn(0x7fd294143e00 0x7fd2940b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.736+0000 7fd29ab0f640 1 -- 172.21.15.72:0/2211409118 >> 172.21.15.72:0/2211409118 conn(0x7fd2940b0a90 msgr2=0x7fd2940b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.736+0000 7fd29ab0f640 1 -- 172.21.15.72:0/2211409118 shutdown_connections 2023-12-12T09:52:49.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.736+0000 7fd29ab0f640 1 -- 172.21.15.72:0/2211409118 wait complete. 2023-12-12T09:52:49.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.738+0000 7fd29ab0f640 1 Processor -- start 2023-12-12T09:52:49.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.738+0000 7fd29ab0f640 1 -- start start 2023-12-12T09:52:49.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd29ab0f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd29406c410 0x7fd29415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd29ab0f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd294140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd29ab0f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd294143e00 0x7fd294141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd29ab0f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29413f2c0 con 0x7fd294072180 2023-12-12T09:52:49.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd29ab0f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2941f7be0 con 0x7fd294143e00 2023-12-12T09:52:49.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd29ab0f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2941f7ec0 con 0x7fd29406c410 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd293fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd294140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd293fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd294140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59118/0 (socket says 172.21.15.72:59118) 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd293fff640 1 -- 172.21.15.72:0/3340133861 learned_addr learned my addr 172.21.15.72:0/3340133861 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd293fff640 1 -- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd29406c410 msgr2=0x7fd29415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd2988ab640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd29406c410 0x7fd29415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd2990ac640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd294143e00 0x7fd294141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd293fff640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd29406c410 0x7fd29415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd293fff640 1 -- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd294143e00 msgr2=0x7fd294141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd293fff640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd294143e00 0x7fd294141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd293fff640 1 -- 172.21.15.72:0/3340133861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280009400 con 0x7fd294072180 2023-12-12T09:52:49.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.739+0000 7fd2990ac640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd294143e00 0x7fd294141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:49.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.740+0000 7fd293fff640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd294140dd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd280009890 tx=0x7fd2800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:49.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.740+0000 7fd2917fa640 1 -- 172.21.15.72:0/3340133861 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28003ba70 con 0x7fd294072180 2023-12-12T09:52:49.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.740+0000 7fd2917fa640 1 -- 172.21.15.72:0/3340133861 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28003bbf0 con 0x7fd294072180 2023-12-12T09:52:49.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.740+0000 7fd2917fa640 1 -- 172.21.15.72:0/3340133861 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280045400 con 0x7fd294072180 2023-12-12T09:52:49.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.740+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2941402f0 con 0x7fd294072180 2023-12-12T09:52:49.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.740+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941408b0 con 0x7fd294072180 2023-12-12T09:52:49.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.741+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2940745d0 con 0x7fd294072180 2023-12-12T09:52:49.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.742+0000 7fd2917fa640 1 -- 172.21.15.72:0/3340133861 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd28004e440 con 0x7fd294072180 2023-12-12T09:52:49.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.743+0000 7fd2917fa640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd27806ae40 0x7fd27806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:49.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.743+0000 7fd2917fa640 1 -- 172.21.15.72:0/3340133861 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd2800abd00 con 0x7fd294072180 2023-12-12T09:52:49.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.743+0000 7fd2988ab640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd27806ae40 0x7fd27806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:49.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.744+0000 7fd2988ab640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd27806ae40 0x7fd27806d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd29413eeb0 tx=0x7fd288008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:49.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.748+0000 7fd2917fa640 1 -- 172.21.15.72:0/3340133861 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd280076a20 con 0x7fd294072180 2023-12-12T09:52:49.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.937+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2940745d0 con 0x7fd294072180 2023-12-12T09:52:49.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.941+0000 7fd2917fa640 1 -- 172.21.15.72:0/3340133861 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fd280035090 con 0x7fd294072180 2023-12-12T09:52:49.943 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:49.943 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:12.519084+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:52:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd27806ae40 msgr2=0x7fd27806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd27806ae40 0x7fd27806d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd29413eeb0 tx=0x7fd288008040 comp rx=0 tx=0).stop 2023-12-12T09:52:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 msgr2=0x7fd294140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd294140dd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd280009890 tx=0x7fd2800046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 shutdown_connections 2023-12-12T09:52:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd27806ae40 0x7fd27806d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd294143e00 0x7fd294141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd294072180 0x7fd294140dd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 --2- 172.21.15.72:0/3340133861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd29406c410 0x7fd29415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 >> 172.21.15.72:0/3340133861 conn(0x7fd2940b0a90 msgr2=0x7fd2940ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 shutdown_connections 2023-12-12T09:52:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:49.943+0000 7fd29ab0f640 1 -- 172.21.15.72:0/3340133861 wait complete. 2023-12-12T09:52:49.964 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:52:50.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.166+0000 7f087ee00640 1 Processor -- start 2023-12-12T09:52:50.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.167+0000 7f087ee00640 1 -- start start 2023-12-12T09:52:50.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.168+0000 7f087ee00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0878143e00 0x7f08780b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:50.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.168+0000 7f087ee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087806c410 0x7f087815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:50.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.168+0000 7f087ee00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08780b3bb0 con 0x7f0878072180 2023-12-12T09:52:50.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.168+0000 7f087ee00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08781441e0 con 0x7f0878143e00 2023-12-12T09:52:50.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.168+0000 7f087ee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0878072680 con 0x7f087806c410 2023-12-12T09:52:50.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.169+0000 7f087cb9c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0878143e00 0x7f08780b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.169+0000 7f087d39d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087806c410 0x7f087815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.169+0000 7f087cb9c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0878143e00 0x7f08780b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:53226/0 (socket says 172.21.15.72:53226) 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.169+0000 7f087cb9c640 1 -- 172.21.15.72:0/3532379778 learned_addr learned my addr 172.21.15.72:0/3532379778 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.170+0000 7f087cb9c640 1 -- 172.21.15.72:0/3532379778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087806c410 msgr2=0x7f087815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.170+0000 7f087cb9c640 1 --2- 172.21.15.72:0/3532379778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087806c410 0x7f087815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.170+0000 7f087cb9c640 1 -- 172.21.15.72:0/3532379778 >> v1:172.21.15.113:6789/0 conn(0x7f0878072180 legacy=0x7f08780b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.170+0000 7f087cb9c640 1 -- 172.21.15.72:0/3532379778 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780b1e30 con 0x7f0878143e00 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.170+0000 7f087d39d640 1 --2- 172.21.15.72:0/3532379778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087806c410 0x7f087815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:50.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.171+0000 7f087cb9c640 1 --2- 172.21.15.72:0/3532379778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0878143e00 0x7f08780b4040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f086c009870 tx=0x7f086c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d1c1b664c26c53c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:50.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.171+0000 7f08777fe640 1 -- 172.21.15.72:0/3532379778 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c030840 con 0x7f0878143e00 2023-12-12T09:52:50.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f08777fe640 1 -- 172.21.15.72:0/3532379778 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c0309a0 con 0x7f0878143e00 2023-12-12T09:52:50.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f08777fe640 1 -- 172.21.15.72:0/3532379778 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c030d30 con 0x7f0878143e00 2023-12-12T09:52:50.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f087ee00640 1 -- 172.21.15.72:0/3532379778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0878143e00 msgr2=0x7f08780b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:50.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f087ee00640 1 --2- 172.21.15.72:0/3532379778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0878143e00 0x7f08780b4040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f086c009870 tx=0x7f086c02b980 comp rx=0 tx=0).stop 2023-12-12T09:52:50.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f087ee00640 1 -- 172.21.15.72:0/3532379778 shutdown_connections 2023-12-12T09:52:50.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f087ee00640 1 --2- 172.21.15.72:0/3532379778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087806c410 0x7f087815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f087ee00640 1 --2- 172.21.15.72:0/3532379778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0878143e00 0x7f08780b4040 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f087ee00640 1 -- 172.21.15.72:0/3532379778 >> 172.21.15.72:0/3532379778 conn(0x7f08780b0a90 msgr2=0x7f08780b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:50.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f087ee00640 1 -- 172.21.15.72:0/3532379778 shutdown_connections 2023-12-12T09:52:50.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.172+0000 7f087ee00640 1 -- 172.21.15.72:0/3532379778 wait complete. 2023-12-12T09:52:50.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.174+0000 7f087ee00640 1 Processor -- start 2023-12-12T09:52:50.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.174+0000 7f087ee00640 1 -- start start 2023-12-12T09:52:50.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.175+0000 7f087ee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087806c410 0x7f087815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:50.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.175+0000 7f087ee00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0878072180 0x7f087815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:50.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.175+0000 7f087ee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0878143e00 0x7f087813d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:50.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.175+0000 7f087ee00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087815fbe0 con 0x7f0878072180 2023-12-12T09:52:50.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.175+0000 7f087ee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087815be00 con 0x7f087806c410 2023-12-12T09:52:50.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.175+0000 7f087ee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087815c0e0 con 0x7f0878143e00 2023-12-12T09:52:50.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0878072180 0x7f087815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:50.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0878072180 0x7f087815f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59124/0 (socket says 172.21.15.72:59124) 2023-12-12T09:52:50.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 -- 172.21.15.72:0/2108126059 learned_addr learned my addr 172.21.15.72:0/2108126059 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:50.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f087cb9c640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087806c410 0x7f087815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:50.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f087d39d640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0878143e00 0x7f087813d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:50.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 -- 172.21.15.72:0/2108126059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0878143e00 msgr2=0x7f087813d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:50.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0878143e00 0x7f087813d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 -- 172.21.15.72:0/2108126059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087806c410 msgr2=0x7f087815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:50.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087806c410 0x7f087815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 -- 172.21.15.72:0/2108126059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0870009a70 con 0x7f0878072180 2023-12-12T09:52:50.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.176+0000 7f0877fff640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0878072180 0x7f087815f6a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f086400c400 tx=0x7f086400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:50.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.177+0000 7f08757fa640 1 -- 172.21.15.72:0/2108126059 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640052d0 con 0x7f0878072180 2023-12-12T09:52:50.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.177+0000 7f08757fa640 1 -- 172.21.15.72:0/2108126059 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864005430 con 0x7f0878072180 2023-12-12T09:52:50.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.177+0000 7f08757fa640 1 -- 172.21.15.72:0/2108126059 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864003830 con 0x7f0878072180 2023-12-12T09:52:50.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.177+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c0094d0 con 0x7f0878072180 2023-12-12T09:52:50.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.177+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087815c6a0 con 0x7f0878072180 2023-12-12T09:52:50.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.178+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08780745d0 con 0x7f0878072180 2023-12-12T09:52:50.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.179+0000 7f08757fa640 1 -- 172.21.15.72:0/2108126059 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0864003990 con 0x7f0878072180 2023-12-12T09:52:50.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.180+0000 7f08757fa640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f085c06ae40 0x7f085c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:50.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.180+0000 7f08757fa640 1 -- 172.21.15.72:0/2108126059 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0864088f30 con 0x7f0878072180 2023-12-12T09:52:50.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.180+0000 7f087cb9c640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f085c06ae40 0x7f085c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:50.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.181+0000 7f087cb9c640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f085c06ae40 0x7f085c06d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f086c036000 tx=0x7f086c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:50.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.184+0000 7f08757fa640 1 -- 172.21.15.72:0/2108126059 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0864053cf0 con 0x7f0878072180 2023-12-12T09:52:50.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.409+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f087806e020 con 0x7f0878072180 2023-12-12T09:52:50.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.410+0000 7f08757fa640 1 -- 172.21.15.72:0/2108126059 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f0864017090 con 0x7f0878072180 2023-12-12T09:52:50.412 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:52:50.412 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:35.924557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:52:50.413 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-12-12T09:52:50.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f085c06ae40 msgr2=0x7f085c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:50.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f085c06ae40 0x7f085c06d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f086c036000 tx=0x7f086c03b040 comp rx=0 tx=0).stop 2023-12-12T09:52:50.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0878072180 msgr2=0x7f087815f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:50.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0878072180 0x7f087815f6a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f086400c400 tx=0x7f086400c8c0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 shutdown_connections 2023-12-12T09:52:50.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f085c06ae40 0x7f085c06d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0878143e00 0x7f087813d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0878072180 0x7f087815f6a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 --2- 172.21.15.72:0/2108126059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087806c410 0x7f087815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:50.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 >> 172.21.15.72:0/2108126059 conn(0x7f08780b0a90 msgr2=0x7f08780af290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:50.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.413+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 shutdown_connections 2023-12-12T09:52:50.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:50.414+0000 7f087ee00640 1 -- 172.21.15.72:0/2108126059 wait complete. 2023-12-12T09:52:50.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-12T09:52:07.436771+0000', 'modified': '2023-12-12T09:52:35.924557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4243}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4243': {'gid': 4243, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.72:6837/1645740011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6836', 'nonce': 1645740011}, {'type': 'v1', 'addr': '172.21.15.72:6837', 'nonce': 1645740011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-12T09:52:50.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-12T09:52:50.836 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-12T09:52:51.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.053+0000 7fad257ea640 1 Processor -- start 2023-12-12T09:52:51.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.053+0000 7fad257ea640 1 -- start start 2023-12-12T09:52:51.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad257ea640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fad20072180 0x7fad200b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad257ea640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad2006c410 0x7fad2015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad257ea640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad200b3bb0 con 0x7fad20143e00 2023-12-12T09:52:51.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad257ea640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad201441e0 con 0x7fad20072180 2023-12-12T09:52:51.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad257ea640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad20072680 con 0x7fad2006c410 2023-12-12T09:52:51.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad1e7fc640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fad20072180 0x7fad200b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad1e7fc640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fad20072180 0x7fad200b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59132/0 (socket says 172.21.15.72:59132) 2023-12-12T09:52:51.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad1e7fc640 1 -- 172.21.15.72:0/13088950 learned_addr learned my addr 172.21.15.72:0/13088950 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:51.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad1f7fe640 1 --2- 172.21.15.72:0/13088950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad2006c410 0x7fad2015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.055+0000 7fad1e7fc640 1 -- 172.21.15.72:0/13088950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad2006c410 msgr2=0x7fad2015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.056+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/13088950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad2006c410 0x7fad2015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.056+0000 7fad1e7fc640 1 -- 172.21.15.72:0/13088950 >> v1:172.21.15.113:6789/0 conn(0x7fad20143e00 legacy=0x7fad200b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.056+0000 7fad1e7fc640 1 -- 172.21.15.72:0/13088950 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200b1e30 con 0x7fad20072180 2023-12-12T09:52:51.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.056+0000 7fad1f7fe640 1 --2- 172.21.15.72:0/13088950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad2006c410 0x7fad2015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:51.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.057+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/13088950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200b4750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad08009870 tx=0x7fad0802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e652a0a3dc19d826 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:51.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.057+0000 7fad1dffb640 1 -- 172.21.15.72:0/13088950 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08031640 con 0x7fad20072180 2023-12-12T09:52:51.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.057+0000 7fad1dffb640 1 -- 172.21.15.72:0/13088950 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad080317a0 con 0x7fad20072180 2023-12-12T09:52:51.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.057+0000 7fad1dffb640 1 -- 172.21.15.72:0/13088950 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08031b30 con 0x7fad20072180 2023-12-12T09:52:51.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.058+0000 7fad257ea640 1 -- 172.21.15.72:0/13088950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 msgr2=0x7fad200b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.058+0000 7fad257ea640 1 --2- 172.21.15.72:0/13088950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200b4750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad08009870 tx=0x7fad0802b8f0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.058+0000 7fad257ea640 1 -- 172.21.15.72:0/13088950 shutdown_connections 2023-12-12T09:52:51.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.058+0000 7fad257ea640 1 --2- 172.21.15.72:0/13088950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad2006c410 0x7fad2015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.058+0000 7fad257ea640 1 --2- 172.21.15.72:0/13088950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200b4750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.058+0000 7fad257ea640 1 -- 172.21.15.72:0/13088950 >> 172.21.15.72:0/13088950 conn(0x7fad200b0a90 msgr2=0x7fad200b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:51.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.058+0000 7fad257ea640 1 -- 172.21.15.72:0/13088950 shutdown_connections 2023-12-12T09:52:51.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.058+0000 7fad257ea640 1 -- 172.21.15.72:0/13088950 wait complete. 2023-12-12T09:52:51.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.060+0000 7fad257ea640 1 Processor -- start 2023-12-12T09:52:51.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.060+0000 7fad257ea640 1 -- start start 2023-12-12T09:52:51.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad257ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad2006c410 0x7fad201f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad257ea640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad257ea640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad20143e00 0x7fad20141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad257ea640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad201f4eb0 con 0x7fad20072180 2023-12-12T09:52:51.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad257ea640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200aee90 con 0x7fad2006c410 2023-12-12T09:52:51.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad257ea640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad201f0d80 con 0x7fad20143e00 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59148/0 (socket says 172.21.15.72:59148) 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1e7fc640 1 -- 172.21.15.72:0/1130305166 learned_addr learned my addr 172.21.15.72:0/1130305166 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1f7fe640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad20143e00 0x7fad20141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1e7fc640 1 -- 172.21.15.72:0/1130305166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad20143e00 msgr2=0x7fad20141f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad20143e00 0x7fad20141f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1e7fc640 1 -- 172.21.15.72:0/1130305166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad2006c410 msgr2=0x7fad201f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad2006c410 0x7fad201f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.061+0000 7fad1e7fc640 1 -- 172.21.15.72:0/1130305166 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080094d0 con 0x7fad20072180 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.062+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200ae950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fad08000c00 tx=0x7fad080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.062+0000 7fad177fe640 1 -- 172.21.15.72:0/1130305166 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0803ac60 con 0x7fad20072180 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.062+0000 7fad177fe640 1 -- 172.21.15.72:0/1130305166 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0803ade0 con 0x7fad20072180 2023-12-12T09:52:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.062+0000 7fad177fe640 1 -- 172.21.15.72:0/1130305166 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08044630 con 0x7fad20072180 2023-12-12T09:52:51.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.062+0000 7fad257ea640 1 -- 172.21.15.72:0/1130305166 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad201f1060 con 0x7fad20072180 2023-12-12T09:52:51.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.062+0000 7fad257ea640 1 -- 172.21.15.72:0/1130305166 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad201f1620 con 0x7fad20072180 2023-12-12T09:52:51.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.065+0000 7fad177fe640 1 -- 172.21.15.72:0/1130305166 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad0802f050 con 0x7fad20072180 2023-12-12T09:52:51.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.066+0000 7fad177fe640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fad0006ae70 0x7fad0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.066+0000 7fad177fe640 1 -- 172.21.15.72:0/1130305166 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fad080aba20 con 0x7fad20072180 2023-12-12T09:52:51.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.067+0000 7fad1effd640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fad0006ae70 0x7fad0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.067+0000 7fad28381740 1 Processor -- start 2023-12-12T09:52:51.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.067+0000 7fad28381740 1 -- start start 2023-12-12T09:52:51.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.067+0000 7fad1effd640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fad0006ae70 0x7fad0006d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad10006fd0 tx=0x7fad10006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:51.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad28381740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f0326420 0x5578f0326800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad28381740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f02f66d0 0x5578f0323690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad28381740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0323bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad28381740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f0264f40 con 0x5578f0326420 2023-12-12T09:52:51.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad28381740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f0262630 con 0x5578f02f9fa0 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad28381740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578f025df20 con 0x5578f02f66d0 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad1effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f0326420 0x5578f0326800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad1effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f0326420 0x5578f0326800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59156/0 (socket says 172.21.15.72:59156) 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.068+0000 7fad1effd640 1 -- 172.21.15.72:0/3928982366 learned_addr learned my addr 172.21.15.72:0/3928982366 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f02f66d0 0x5578f0323690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1effd640 1 -- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f02f66d0 msgr2=0x5578f0323690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1effd640 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f02f66d0 0x5578f0323690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1f7fe640 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0323bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1effd640 1 -- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 msgr2=0x5578f0323bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1effd640 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0323bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1effd640 1 -- 172.21.15.72:0/3928982366 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578f0220910 con 0x5578f0326420 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f02f66d0 0x5578f0323690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad1effd640 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f0326420 0x5578f0326800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad100138c0 tx=0x7fad1000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d71927760fe9ce server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad14ff9640 1 -- 172.21.15.72:0/3928982366 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10012ca0 con 0x5578f0326420 2023-12-12T09:52:51.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad14ff9640 1 -- 172.21.15.72:0/3928982366 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10019040 con 0x5578f0326420 2023-12-12T09:52:51.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.069+0000 7fad14ff9640 1 -- 172.21.15.72:0/3928982366 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10013e40 con 0x5578f0326420 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 -- 172.21.15.72:0/3928982366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f0326420 msgr2=0x5578f0326800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f0326420 0x5578f0326800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad100138c0 tx=0x7fad1000e000 comp rx=0 tx=0).stop 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 -- 172.21.15.72:0/3928982366 shutdown_connections 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0323bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f02f66d0 0x5578f0323690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 --2- 172.21.15.72:0/3928982366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f0326420 0x5578f0326800 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 -- 172.21.15.72:0/3928982366 >> 172.21.15.72:0/3928982366 conn(0x5578f02f8940 msgr2=0x5578f0341230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 -- 172.21.15.72:0/3928982366 shutdown_connections 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.070+0000 7fad28381740 1 -- 172.21.15.72:0/3928982366 wait complete. 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.071+0000 7fad28381740 1 -- start start 2023-12-12T09:52:51.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.071+0000 7fad28381740 1 Processor -- start 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.072+0000 7fad28381740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0341670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.072+0000 7fad28381740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f0310fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.072+0000 7fad28381740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0320ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.072+0000 7fad28381740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f01fa2f0 con 0x5578f031f500 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.072+0000 7fad28381740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f0217c70 con 0x5578f02f9fa0 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.072+0000 7fad28381740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578effc0f20 con 0x5578f0326420 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.072+0000 7fad28381740 10 client.0 _ll_register_callbacks cb 0x5578f021c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0320ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0320ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59170/0 (socket says 172.21.15.72:59170) 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 -- 172.21.15.72:0/533382162 learned_addr learned my addr 172.21.15.72:0/533382162 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1effd640 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0341670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f0310fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 -- 172.21.15.72:0/533382162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 msgr2=0x5578f0310fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f0310fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 -- 172.21.15.72:0/533382162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 msgr2=0x5578f0341670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0341670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 -- 172.21.15.72:0/533382162 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad10013570 con 0x5578f031f500 2023-12-12T09:52:51.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7fad1f7fe640 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0320ff0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad1800cb20 tx=0x7fad18008320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e5188187c60d30 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:51.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.073+0000 7facfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2023-12-12T09:52:51.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7facfdffb640 1 -- 172.21.15.72:0/533382162 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011070 con 0x5578f031f500 2023-12-12T09:52:51.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 1 -- 172.21.15.72:0/533382162 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578f0256db0 con 0x5578f031f500 2023-12-12T09:52:51.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7facfdffb640 1 -- 172.21.15.72:0/533382162 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fad18017dc0 con 0x5578f031f500 2023-12-12T09:52:51.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7facfdffb640 1 -- 172.21.15.72:0/533382162 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad18009a70 con 0x5578f031f500 2023-12-12T09:52:51.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 10 client.4827 fetch_fsmap learned FSMap version 8 2023-12-12T09:52:51.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 1 -- 172.21.15.72:0/533382162 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5578f025c5a0 con 0x5578f031f500 2023-12-12T09:52:51.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7facfdffb640 1 -- 172.21.15.72:0/533382162 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5578f0256db0 con 0x5578f031f500 2023-12-12T09:52:51.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7facfdffb640 1 -- 172.21.15.72:0/533382162 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fad18008040 con 0x5578f031f500 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 10 client.4827 fetch_fsmap finished waiting for FSMap version 8 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 10 client.4827 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 20 client.4827 populate_metadata read hostname 'smithi072' 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x5578f0327db0 0x5578f0328190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 4 client.4827 mds_command: new command op to 4243 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.074+0000 7fad28381740 1 -- 172.21.15.72:0/533382162 --> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578f0256ad0 con 0x5578f0327db0 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.075+0000 7fad1effd640 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x5578f0327db0 0x5578f0328190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.075+0000 7fad1effd640 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x5578f0327db0 0x5578f0328190 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c24c8935547d06f server_cookie=9cbe6f524205d1ac in_seq=0 out_seq=0 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.075+0000 7facfdffb640 10 client.4827 ms_handle_connect on v2:172.21.15.72:6836/1645740011 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7facfdffb640 1 -- 172.21.15.72:0/533382162 <== mds.0 v2:172.21.15.72:6836/1645740011 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578f0256ad0 con 0x5578f0327db0 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7facfdffb640 10 client.4827 handle_command_reply: tid=0 2023-12-12T09:52:51.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 client.4827 shutdown 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 -- 172.21.15.72:0/533382162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 msgr2=0x5578f0320ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0320ff0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad1800cb20 tx=0x7fad18008320 comp rx=0 tx=0).stop 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 -- 172.21.15.72:0/533382162 shutdown_connections 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x5578f0327db0 0x5578f0328190 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0320ff0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f0310fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 --2- 172.21.15.72:0/533382162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0341670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 -- 172.21.15.72:0/533382162 >> 172.21.15.72:0/533382162 conn(0x5578f02f66d0 msgr2=0x5578f02fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7facfdffb640 0 client.4827 ms_handle_reset on v2:172.21.15.72:6836/1645740011 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 -- 172.21.15.72:0/533382162 shutdown_connections 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.076+0000 7fad28381740 1 -- 172.21.15.72:0/533382162 wait complete. 2023-12-12T09:52:51.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.077+0000 7fad28381740 20 client.4827 trim_cache size 0 max 16384 2023-12-12T09:52:51.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.093+0000 7fad28381740 1 Processor -- start 2023-12-12T09:52:51.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.093+0000 7fad28381740 1 -- start start 2023-12-12T09:52:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.094+0000 7fad28381740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0344040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.094+0000 7fad28381740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0321de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.094+0000 7fad28381740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f0363780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.094+0000 7fad28381740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f02f7610 con 0x5578f031f500 2023-12-12T09:52:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.094+0000 7fad28381740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f02f6d00 con 0x5578f02f9fa0 2023-12-12T09:52:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.094+0000 7fad28381740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578f031fb10 con 0x5578f0326420 2023-12-12T09:52:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.094+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0321de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0321de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59172/0 (socket says 172.21.15.72:59172) 2023-12-12T09:52:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1e7fc640 1 -- 172.21.15.72:0/1177302561 learned_addr learned my addr 172.21.15.72:0/1177302561 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:51.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1effd640 1 --2- 172.21.15.72:0/1177302561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0344040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1e7fc640 1 -- 172.21.15.72:0/1177302561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 msgr2=0x5578f0363780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/1177302561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f0363780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1e7fc640 1 -- 172.21.15.72:0/1177302561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 msgr2=0x5578f0344040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/1177302561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0344040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1e7fc640 1 -- 172.21.15.72:0/1177302561 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578f02f8050 con 0x5578f031f500 2023-12-12T09:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/1177302561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0321de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fad08076990 tx=0x7fad08076900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ff2d14efd51f49 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad157fa640 1 -- 172.21.15.72:0/1177302561 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0807a490 con 0x5578f031f500 2023-12-12T09:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad157fa640 1 -- 172.21.15.72:0/1177302561 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0807e5e0 con 0x5578f031f500 2023-12-12T09:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.095+0000 7fad157fa640 1 -- 172.21.15.72:0/1177302561 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08082790 con 0x5578f031f500 2023-12-12T09:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 -- 172.21.15.72:0/1177302561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 msgr2=0x5578f0321de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 --2- 172.21.15.72:0/1177302561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0321de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fad08076990 tx=0x7fad08076900 comp rx=0 tx=0).stop 2023-12-12T09:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 -- 172.21.15.72:0/1177302561 shutdown_connections 2023-12-12T09:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 --2- 172.21.15.72:0/1177302561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f0363780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 --2- 172.21.15.72:0/1177302561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f031f500 0x5578f0321de0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 --2- 172.21.15.72:0/1177302561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f02f9fa0 0x5578f0344040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 -- 172.21.15.72:0/1177302561 >> 172.21.15.72:0/1177302561 conn(0x5578f02f66d0 msgr2=0x5578f0341cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 -- 172.21.15.72:0/1177302561 shutdown_connections 2023-12-12T09:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.096+0000 7fad28381740 1 -- 172.21.15.72:0/1177302561 wait complete. 2023-12-12T09:52:51.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.097+0000 7fad28381740 1 -- start start 2023-12-12T09:52:51.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.097+0000 7fad28381740 1 Processor -- start 2023-12-12T09:52:51.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad28381740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f02f9fa0 0x5578f03406e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad28381740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f031f500 0x5578f03aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad28381740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f03aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad28381740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f03116f0 con 0x5578f02f9fa0 2023-12-12T09:52:51.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad28381740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f0329a60 con 0x5578f031f500 2023-12-12T09:52:51.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad28381740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578f0340c20 con 0x5578f0326420 2023-12-12T09:52:51.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad28381740 10 client.0 _ll_register_callbacks cb 0x5578f02f20c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:52:51.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad1effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f02f9fa0 0x5578f03406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad1effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f02f9fa0 0x5578f03406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59188/0 (socket says 172.21.15.72:59188) 2023-12-12T09:52:51.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad1effd640 1 -- 172.21.15.72:0/745108647 learned_addr learned my addr 172.21.15.72:0/745108647 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:51.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f031f500 0x5578f03aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad1effd640 1 -- 172.21.15.72:0/745108647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 msgr2=0x5578f03aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.098+0000 7fad1effd640 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f03aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7fad1effd640 1 -- 172.21.15.72:0/745108647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f031f500 msgr2=0x5578f03aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7fad1effd640 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f031f500 0x5578f03aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7fad1effd640 1 -- 172.21.15.72:0/745108647 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad201f1620 con 0x5578f02f9fa0 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f031f500 0x5578f03aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7fad1effd640 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f02f9fa0 0x5578f03406e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad1001e000 tx=0x7fad10004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc2d052deed2069 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7facfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7facfdffb640 1 -- 172.21.15.72:0/745108647 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1002c040 con 0x5578f02f9fa0 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7facfdffb640 1 -- 172.21.15.72:0/745108647 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fad10013e40 con 0x5578f02f9fa0 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7fad28381740 1 -- 172.21.15.72:0/745108647 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad10019040 con 0x5578f02f9fa0 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7facfdffb640 1 -- 172.21.15.72:0/745108647 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad10012c20 con 0x5578f02f9fa0 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7fad28381740 10 client.4833 fetch_fsmap learned FSMap version 8 2023-12-12T09:52:51.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.099+0000 7fad28381740 1 -- 172.21.15.72:0/745108647 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x7fad200ce970 con 0x5578f02f9fa0 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7facfdffb640 1 -- 172.21.15.72:0/745108647 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10019040 con 0x5578f02f9fa0 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7facfdffb640 1 -- 172.21.15.72:0/745108647 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fad1002f030 con 0x5578f02f9fa0 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7fad28381740 10 client.4833 fetch_fsmap finished waiting for FSMap version 8 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7fad28381740 10 client.4833 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7fad28381740 20 client.4833 populate_metadata read hostname 'smithi072' 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7fad28381740 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x5578f0310a20 0x5578f0310e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7fad28381740 4 client.4833 mds_command: new command op to 4243 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7fad28381740 1 -- 172.21.15.72:0/745108647 --> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5578eff5e6c0 con 0x5578f0310a20 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.100+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x5578f0310a20 0x5578f0310e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad1e7fc640 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x5578f0310a20 0x5578f0310e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72951b8cec34db89 server_cookie=b543106ab6cc470d in_seq=0 out_seq=0 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7facfdffb640 10 client.4833 ms_handle_connect on v2:172.21.15.72:6836/1645740011 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7facfdffb640 1 -- 172.21.15.72:0/745108647 <== mds.0 v2:172.21.15.72:6836/1645740011 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5578eff5e6c0 con 0x5578f0310a20 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7facfdffb640 10 client.4833 handle_command_reply: tid=0 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 client.4833 shutdown 2023-12-12T09:52:51.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 -- 172.21.15.72:0/745108647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f02f9fa0 msgr2=0x5578f03406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f02f9fa0 0x5578f03406e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad1001e000 tx=0x7fad10004760 comp rx=0 tx=0).stop 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 -- 172.21.15.72:0/745108647 shutdown_connections 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.72:6836/1645740011,v1:172.21.15.72:6837/1645740011] conn(0x5578f0310a20 0x5578f0310e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578f0326420 0x5578f03aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578f031f500 0x5578f03aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 --2- 172.21.15.72:0/745108647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578f02f9fa0 0x5578f03406e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7facfdffb640 0 client.4833 ms_handle_reset on v2:172.21.15.72:6836/1645740011 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 -- 172.21.15.72:0/745108647 >> 172.21.15.72:0/745108647 conn(0x5578f02f66d0 msgr2=0x5578f0321de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 -- 172.21.15.72:0/745108647 shutdown_connections 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.101+0000 7fad28381740 1 -- 172.21.15.72:0/745108647 wait complete. 2023-12-12T09:52:51.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad28381740 20 client.4833 trim_cache size 0 max 16384 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "bd07f350-42f2-4d45-9a3d-13364a8566fe", 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 4243, 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 43.081939143, 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 8, 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 24, 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 24, 2023-12-12T09:52:51.110 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 50.315331424999997 2023-12-12T09:52:51.111 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:52:51.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 -- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fad0006ae70 msgr2=0x7fad0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fad0006ae70 0x7fad0006d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad10006fd0 tx=0x7fad10006bc0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 -- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 msgr2=0x7fad200ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200ae950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fad08000c00 tx=0x7fad080046c0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 -- 172.21.15.72:0/1130305166 shutdown_connections 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fad0006ae70 0x7fad0006d330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad20143e00 0x7fad20141f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad20072180 0x7fad200ae950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 --2- 172.21.15.72:0/1130305166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad2006c410 0x7fad201f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 -- 172.21.15.72:0/1130305166 >> 172.21.15.72:0/1130305166 conn(0x7fad200b0a90 msgr2=0x7fad200af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 -- 172.21.15.72:0/1130305166 shutdown_connections 2023-12-12T09:52:51.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:51.102+0000 7fad257ea640 1 -- 172.21.15.72:0/1130305166 wait complete. 2023-12-12T09:52:51.122 INFO:teuthology.run_tasks:Running task kclient... 2023-12-12T09:52:51.141 INFO:tasks.kclient:Mounting kernel clients... 2023-12-12T09:52:51.141 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-12T09:52:51.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.142 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:52:51.199 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-12-12T09:52:51.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.200 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:52:51.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.294 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2023-12-12T09:52:51.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.418 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:52:51.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.464 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2023-12-12T09:52:51.601 INFO:teuthology.orchestra.run.smithi072.stderr:Cannot find device "ceph-brx" 2023-12-12T09:52:51.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:52:51.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.605 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-12-12T09:52:51.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2023-12-12T09:52:51.762 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2023-12-12T09:52:51.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:52:51.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.768 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-12-12T09:52:51.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:51.822 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2023-12-12T09:52:51.915 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2023-12-12T09:52:51.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:52:51.919 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-12-12T09:52:51.919 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-12T09:52:51.919 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-12T09:52:51.919 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-12T09:52:52.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.147+0000 7f086b2a3640 1 Processor -- start 2023-12-12T09:52:52.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.148+0000 7f086b2a3640 1 -- start start 2023-12-12T09:52:52.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.149+0000 7f086b2a3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f086415e5d0 0x7f086406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:52.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.149+0000 7f086b2a3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08640ae200 0x7f0864070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:52.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.149+0000 7f086b2a3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f0864071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:52.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.149+0000 7f086b2a3640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0864071f90 con 0x7f086415e5d0 2023-12-12T09:52:52.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.149+0000 7f086b2a3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0864072270 con 0x7f08640ae200 2023-12-12T09:52:52.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.149+0000 7f086b2a3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0864070070 con 0x7f086406c330 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.149+0000 7f086a2a1640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f086415e5d0 0x7f086406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086a2a1640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f086415e5d0 0x7f086406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59204/0 (socket says 172.21.15.72:59204) 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086a2a1640 1 -- 172.21.15.72:0/3175434262 learned_addr learned my addr 172.21.15.72:0/3175434262 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086aaa2640 1 --2- 172.21.15.72:0/3175434262 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f0864071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f0869aa0640 1 --2- 172.21.15.72:0/3175434262 >> v2:172.21.15.113:3300/0 conn(0x7f08640ae200 0x7f0864070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086a2a1640 1 -- 172.21.15.72:0/3175434262 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 msgr2=0x7f0864071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086a2a1640 1 --2- 172.21.15.72:0/3175434262 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f0864071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086a2a1640 1 -- 172.21.15.72:0/3175434262 >> v2:172.21.15.113:3300/0 conn(0x7f08640ae200 msgr2=0x7f0864070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086a2a1640 1 --2- 172.21.15.72:0/3175434262 >> v2:172.21.15.113:3300/0 conn(0x7f08640ae200 0x7f0864070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086a2a1640 1 -- 172.21.15.72:0/3175434262 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0864070350 con 0x7f086415e5d0 2023-12-12T09:52:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.150+0000 7f086aaa2640 1 --2- 172.21.15.72:0/3175434262 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f0864071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.151+0000 7f086a2a1640 1 --2- 172.21.15.72:0/3175434262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f086415e5d0 0x7f086406bdf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f08640739e0 tx=0x7f086002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ed5a2ed5c5ee02 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.152+0000 7f086929f640 1 -- 172.21.15.72:0/3175434262 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860031750 con 0x7f086415e5d0 2023-12-12T09:52:52.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.152+0000 7f086929f640 1 -- 172.21.15.72:0/3175434262 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08600318b0 con 0x7f086415e5d0 2023-12-12T09:52:52.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.152+0000 7f086929f640 1 -- 172.21.15.72:0/3175434262 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860031c40 con 0x7f086415e5d0 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.152+0000 7f086b2a3640 1 -- 172.21.15.72:0/3175434262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f086415e5d0 msgr2=0x7f086406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.152+0000 7f086b2a3640 1 --2- 172.21.15.72:0/3175434262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f086415e5d0 0x7f086406bdf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f08640739e0 tx=0x7f086002b7f0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.153+0000 7f086b2a3640 1 -- 172.21.15.72:0/3175434262 shutdown_connections 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.153+0000 7f086b2a3640 1 --2- 172.21.15.72:0/3175434262 >> v2:172.21.15.113:3300/0 conn(0x7f08640ae200 0x7f0864070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.153+0000 7f086b2a3640 1 --2- 172.21.15.72:0/3175434262 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f0864071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.153+0000 7f086b2a3640 1 --2- 172.21.15.72:0/3175434262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f086415e5d0 0x7f086406bdf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.153+0000 7f086b2a3640 1 -- 172.21.15.72:0/3175434262 >> 172.21.15.72:0/3175434262 conn(0x7f08640b0860 msgr2=0x7f08640b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.153+0000 7f086b2a3640 1 -- 172.21.15.72:0/3175434262 shutdown_connections 2023-12-12T09:52:52.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.153+0000 7f086b2a3640 1 -- 172.21.15.72:0/3175434262 wait complete. 2023-12-12T09:52:52.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.155+0000 7f086b2a3640 1 Processor -- start 2023-12-12T09:52:52.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.155+0000 7f086b2a3640 1 -- start start 2023-12-12T09:52:52.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.156+0000 7f086b2a3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f08641f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:52.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.156+0000 7f086b2a3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08640ae200 0x7f086414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:52.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.156+0000 7f086b2a3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086415e5d0 0x7f08641506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:52.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f086b2a3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086409a320 con 0x7f08640ae200 2023-12-12T09:52:52.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f086b2a3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086414ed00 con 0x7f086415e5d0 2023-12-12T09:52:52.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f086b2a3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086414fe20 con 0x7f086406c330 2023-12-12T09:52:52.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f0869aa0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08640ae200 0x7f086414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:52.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f0869aa0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08640ae200 0x7f086414e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59206/0 (socket says 172.21.15.72:59206) 2023-12-12T09:52:52.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f0869aa0640 1 -- 172.21.15.72:0/769701748 learned_addr learned my addr 172.21.15.72:0/769701748 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f086aaa2640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086415e5d0 0x7f08641506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f086a2a1640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f08641f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f0869aa0640 1 -- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 msgr2=0x7f08641f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f0869aa0640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f08641f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f0869aa0640 1 -- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086415e5d0 msgr2=0x7f08641506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f0869aa0640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086415e5d0 0x7f08641506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f0869aa0640 1 -- 172.21.15.72:0/769701748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08600094d0 con 0x7f08640ae200 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.157+0000 7f086aaa2640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086415e5d0 0x7f08641506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.158+0000 7f0869aa0640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08640ae200 0x7f086414e7c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f085400d830 tx=0x7f085400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.158+0000 7f085affd640 1 -- 172.21.15.72:0/769701748 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400b6c0 con 0x7f08640ae200 2023-12-12T09:52:52.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.158+0000 7f085affd640 1 -- 172.21.15.72:0/769701748 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085400b820 con 0x7f08640ae200 2023-12-12T09:52:52.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.158+0000 7f085affd640 1 -- 172.21.15.72:0/769701748 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400bbb0 con 0x7f08640ae200 2023-12-12T09:52:52.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.158+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08640999f0 con 0x7f08640ae200 2023-12-12T09:52:52.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.158+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0864099fb0 con 0x7f08640ae200 2023-12-12T09:52:52.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.159+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086409a4d0 con 0x7f08640ae200 2023-12-12T09:52:52.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.160+0000 7f085affd640 1 -- 172.21.15.72:0/769701748 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f085401e460 con 0x7f08640ae200 2023-12-12T09:52:52.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.161+0000 7f085affd640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f084806ae40 0x7f084806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:52.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.161+0000 7f085affd640 1 -- 172.21.15.72:0/769701748 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f085408a0b0 con 0x7f08640ae200 2023-12-12T09:52:52.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.161+0000 7f086a2a1640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f084806ae40 0x7f084806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:52.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.162+0000 7f086a2a1640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f084806ae40 0x7f084806d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0860035000 tx=0x7f0860008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:52.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.166+0000 7f085affd640 1 -- 172.21.15.72:0/769701748 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f086409a4d0 con 0x7f08640ae200 2023-12-12T09:52:52.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.394+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0864074450 con 0x7f08640ae200 2023-12-12T09:52:52.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.395+0000 7f085affd640 1 -- 172.21.15.72:0/769701748 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0854054dd0 con 0x7f08640ae200 2023-12-12T09:52:52.397 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-12T09:52:52.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f084806ae40 msgr2=0x7f084806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:52.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f084806ae40 0x7f084806d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0860035000 tx=0x7f0860008030 comp rx=0 tx=0).stop 2023-12-12T09:52:52.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08640ae200 msgr2=0x7f086414e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08640ae200 0x7f086414e7c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f085400d830 tx=0x7f085400dcf0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 shutdown_connections 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f084806ae40 0x7f084806d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086415e5d0 0x7f08641506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08640ae200 0x7f086414e7c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 --2- 172.21.15.72:0/769701748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086406c330 0x7f08641f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 >> 172.21.15.72:0/769701748 conn(0x7f08640b0860 msgr2=0x7f086406cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 shutdown_connections 2023-12-12T09:52:52.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:52.398+0000 7f086b2a3640 1 -- 172.21.15.72:0/769701748 wait complete. 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:52:52.418 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:52:52.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:52.419 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2023-12-12T09:52:52.474 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:52:52.474 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:52:52.474 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-12T09:52:52.475 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 42330sec preferred_lft 42330sec 2023-12-12T09:52:52.476 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2023-12-12T09:52:52.476 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:52.476 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-12T09:52:52.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:52.477 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:52.477 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:52.477 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add name ceph-brx type bridge 2023-12-12T09:52:52.477 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr flush dev ceph-brx 2023-12-12T09:52:52.477 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set ceph-brx up 2023-12-12T09:52:52.477 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-12T09:52:52.477 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:52.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:52.703 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:52:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-12-12T09:52:52.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:52.784 DEBUG:teuthology.orchestra.run.smithi072:> route 2023-12-12T09:52:52.837 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2023-12-12T09:52:52.838 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:52:52.838 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-12T09:52:52.838 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-12T09:52:52.838 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:52:52.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:52.838 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:52.838 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:52.838 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-12T09:52:52.838 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-12T09:52:52.838 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-12T09:52:52.838 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:53.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:53.014 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:52:53.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:53.065 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2023-12-12T09:52:53.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:53.120 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:53.120 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:53.120 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-12T09:52:53.120 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-12T09:52:53.120 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:53.258 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-12T09:52:53.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:53.259 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:53.259 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:53.259 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-12T09:52:53.259 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-12T09:52:53.259 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-12T09:52:53.259 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-12T09:52:53.259 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-12T09:52:53.259 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:53.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:53.515 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:53.515 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:53.515 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2023-12-12T09:52:53.515 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.0 master ceph-brx 2023-12-12T09:52:53.515 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:53.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:52:53.677 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-12T09:52:53.724 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:52:53.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:52:53.725 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-12T09:52:53.771 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-12T09:52:53.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:53.772 DEBUG:teuthology.orchestra.run.smithi072:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:52:53.961 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:52:53.962 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:52:54.674 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:52:54.674 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:52:54.674 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.72:3300,172.21.15.113:3300,172.21.15.113:3301" 2023-12-12T09:52:54.674 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with new device syntax: 0@bd07f350-42f2-4d45-9a3d-13364a8566fe.cephfs=/ 2023-12-12T09:52:54.674 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.72:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2023-12-12T09:52:54.677 INFO:teuthology.orchestra.run.smithi072.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-12T09:52:54.677 INFO:teuthology.orchestra.run.smithi072.stdout: You just mounted a file system that supports labels which does not 2023-12-12T09:52:54.677 INFO:teuthology.orchestra.run.smithi072.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-12T09:52:54.677 INFO:teuthology.orchestra.run.smithi072.stdout: applications will generate AVC messages and not be allowed access to 2023-12-12T09:52:54.677 INFO:teuthology.orchestra.run.smithi072.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-12T09:52:54.681 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-12T09:52:54.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:54.681 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-12T09:52:54.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:54.772 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:52:54.826 INFO:teuthology.orchestra.run.smithi072.stdout:client4842 2023-12-12T09:52:54.827 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:52:54.827 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:52:54.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:54.876 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:52:54.979 INFO:teuthology.orchestra.run.smithi072.stdout:client4842 2023-12-12T09:52:54.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:54.980 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:52:55.038 INFO:teuthology.orchestra.run.smithi072.stdout:client4842 2023-12-12T09:52:55.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:55.039 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/bd07f350-42f2-4d45-9a3d-13364a8566fe.client4842/status') 2023-12-12T09:52:55.126 INFO:teuthology.orchestra.run.smithi072.stdout:instance: client.4842 (3)192.168.0.1:0/2064971444 2023-12-12T09:52:55.126 INFO:teuthology.orchestra.run.smithi072.stdout:blocklisted: false 2023-12-12T09:52:55.127 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records in 2023-12-12T09:52:55.127 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records out 2023-12-12T09:52:55.127 INFO:teuthology.orchestra.run.smithi072.stderr:69 bytes copied, 7.2109e-05 s, 957 kB/s 2023-12-12T09:52:55.130 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-12-12T09:52:55.130 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-12T09:52:55.130 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-12T09:52:55.130 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-12T09:52:55.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.333+0000 7f70927c9640 1 Processor -- start 2023-12-12T09:52:55.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.333+0000 7f70927c9640 1 -- start start 2023-12-12T09:52:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.334+0000 7f70927c9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f708c143e00 0x7f708c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.334+0000 7f70927c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c06c410 0x7f708c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.334+0000 7f70927c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708c0b3bb0 con 0x7f708c072180 2023-12-12T09:52:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.334+0000 7f70927c9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708c1441e0 con 0x7f708c143e00 2023-12-12T09:52:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.334+0000 7f70927c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c072680 con 0x7f708c06c410 2023-12-12T09:52:55.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.335+0000 7f708bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f708c143e00 0x7f708c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:55.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.335+0000 7f7090d66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c06c410 0x7f708c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:55.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.335+0000 7f708bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f708c143e00 0x7f708c0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:54884/0 (socket says 172.21.15.72:54884) 2023-12-12T09:52:55.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.335+0000 7f708bfff640 1 -- 172.21.15.72:0/3715278577 learned_addr learned my addr 172.21.15.72:0/3715278577 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:55.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.336+0000 7f708bfff640 1 -- 172.21.15.72:0/3715278577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c06c410 msgr2=0x7f708c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:55.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.336+0000 7f708bfff640 1 --2- 172.21.15.72:0/3715278577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c06c410 0x7f708c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.336+0000 7f708bfff640 1 -- 172.21.15.72:0/3715278577 >> v1:172.21.15.113:6789/0 conn(0x7f708c072180 legacy=0x7f708c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:55.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.336+0000 7f708bfff640 1 -- 172.21.15.72:0/3715278577 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0b1e30 con 0x7f708c143e00 2023-12-12T09:52:55.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.336+0000 7f7090d66640 1 --2- 172.21.15.72:0/3715278577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c06c410 0x7f708c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:55.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.338+0000 7f708bfff640 1 --2- 172.21.15.72:0/3715278577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c143e00 0x7f708c0b4040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f707c009870 tx=0x7f707c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61a7bd53109dacb server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:55.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.338+0000 7f708b7fe640 1 -- 172.21.15.72:0/3715278577 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c030840 con 0x7f708c143e00 2023-12-12T09:52:55.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f708b7fe640 1 -- 172.21.15.72:0/3715278577 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c0309a0 con 0x7f708c143e00 2023-12-12T09:52:55.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f708b7fe640 1 -- 172.21.15.72:0/3715278577 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c030d30 con 0x7f708c143e00 2023-12-12T09:52:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f70927c9640 1 -- 172.21.15.72:0/3715278577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c143e00 msgr2=0x7f708c0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f70927c9640 1 --2- 172.21.15.72:0/3715278577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c143e00 0x7f708c0b4040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f707c009870 tx=0x7f707c02b980 comp rx=0 tx=0).stop 2023-12-12T09:52:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f70927c9640 1 -- 172.21.15.72:0/3715278577 shutdown_connections 2023-12-12T09:52:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f70927c9640 1 --2- 172.21.15.72:0/3715278577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c06c410 0x7f708c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f70927c9640 1 --2- 172.21.15.72:0/3715278577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c143e00 0x7f708c0b4040 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f70927c9640 1 -- 172.21.15.72:0/3715278577 >> 172.21.15.72:0/3715278577 conn(0x7f708c0b0a90 msgr2=0x7f708c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f70927c9640 1 -- 172.21.15.72:0/3715278577 shutdown_connections 2023-12-12T09:52:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.339+0000 7f70927c9640 1 -- 172.21.15.72:0/3715278577 wait complete. 2023-12-12T09:52:55.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.341+0000 7f70927c9640 1 Processor -- start 2023-12-12T09:52:55.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.341+0000 7f70927c9640 1 -- start start 2023-12-12T09:52:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.342+0000 7f70927c9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f708c06c410 0x7f708c12f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.342+0000 7f70927c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c072180 0x7f708c12fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.342+0000 7f70927c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c143e00 0x7f708c141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.342+0000 7f70927c9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1f37d0 con 0x7f708c06c410 2023-12-12T09:52:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.342+0000 7f70927c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c130d50 con 0x7f708c072180 2023-12-12T09:52:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.342+0000 7f70927c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1304f0 con 0x7f708c143e00 2023-12-12T09:52:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f708bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f708c06c410 0x7f708c12f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f708bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f708c06c410 0x7f708c12f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60712/0 (socket says 172.21.15.72:60712) 2023-12-12T09:52:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f708bfff640 1 -- 172.21.15.72:0/2520172979 learned_addr learned my addr 172.21.15.72:0/2520172979 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f7090d66640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c143e00 0x7f708c141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f7083fff640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c072180 0x7f708c12fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f708bfff640 1 -- 172.21.15.72:0/2520172979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c143e00 msgr2=0x7f708c141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f708bfff640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c143e00 0x7f708c141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f708bfff640 1 -- 172.21.15.72:0/2520172979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c072180 msgr2=0x7f708c12fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f708bfff640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c072180 0x7f708c12fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.343+0000 7f708bfff640 1 -- 172.21.15.72:0/2520172979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7084009a70 con 0x7f708c06c410 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.344+0000 7f708bfff640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f708c06c410 0x7f708c12f980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.344+0000 7f70897fa640 1 -- 172.21.15.72:0/2520172979 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c039a90 con 0x7f708c06c410 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.344+0000 7f70897fa640 1 -- 172.21.15.72:0/2520172979 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c039c10 con 0x7f708c06c410 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.344+0000 7f70897fa640 1 -- 172.21.15.72:0/2520172979 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c043410 con 0x7f708c06c410 2023-12-12T09:52:55.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.344+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c0094d0 con 0x7f708c06c410 2023-12-12T09:52:55.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.344+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c1f3d20 con 0x7f708c06c410 2023-12-12T09:52:55.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.345+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c0745d0 con 0x7f708c06c410 2023-12-12T09:52:55.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.346+0000 7f70897fa640 1 -- 172.21.15.72:0/2520172979 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f707c02f390 con 0x7f708c06c410 2023-12-12T09:52:55.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.347+0000 7f70897fa640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f706c06ae40 0x7f706c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:55.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.347+0000 7f70897fa640 1 -- 172.21.15.72:0/2520172979 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f707c0ab9a0 con 0x7f708c06c410 2023-12-12T09:52:55.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.347+0000 7f7083fff640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f706c06ae40 0x7f706c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:55.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.348+0000 7f7083fff640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f706c06ae40 0x7f706c06d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f708c067720 tx=0x7f7074009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:55.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.352+0000 7f70897fa640 1 -- 172.21.15.72:0/2520172979 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f707c076720 con 0x7f708c06c410 2023-12-12T09:52:55.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.594+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f708c06ddc0 con 0x7f708c06c410 2023-12-12T09:52:55.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.594+0000 7f70897fa640 1 -- 172.21.15.72:0/2520172979 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f707c079d40 con 0x7f708c06c410 2023-12-12T09:52:55.596 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-12T09:52:55.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f706c06ae40 msgr2=0x7f706c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:55.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f706c06ae40 0x7f706c06d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f708c067720 tx=0x7f7074009040 comp rx=0 tx=0).stop 2023-12-12T09:52:55.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f708c06c410 msgr2=0x7f708c12f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:55.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f708c06c410 0x7f708c12f980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c004730 comp rx=0 tx=0).stop 2023-12-12T09:52:55.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 shutdown_connections 2023-12-12T09:52:55.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f706c06ae40 0x7f706c06d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f708c143e00 0x7f708c141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c072180 0x7f708c12fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 --2- 172.21.15.72:0/2520172979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f708c06c410 0x7f708c12f980 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:55.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 >> 172.21.15.72:0/2520172979 conn(0x7f708c0b0a90 msgr2=0x7f708c144320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:55.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.597+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 shutdown_connections 2023-12-12T09:52:55.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:55.598+0000 7f70927c9640 1 -- 172.21.15.72:0/2520172979 wait complete. 2023-12-12T09:52:55.617 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-12T09:52:55.617 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:52:55.617 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2023-12-12T09:52:55.617 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-12T09:52:55.618 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-12T09:52:55.618 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-12T09:52:55.618 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-12T09:52:55.618 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:52:55.618 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-12T09:52:55.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:55.618 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2023-12-12T09:52:55.671 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 42327sec preferred_lft 42327sec 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 3e:b1:ce:9c:47:2c brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-12T09:52:55.672 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:55.673 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::c4c0:7aff:fe81:b145/64 scope link 2023-12-12T09:52:55.673 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:55.673 INFO:teuthology.orchestra.run.smithi072.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-12T09:52:55.673 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 3e:b1:ce:9c:47:2c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-12T09:52:55.673 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::3cb1:ceff:fe9c:472c/64 scope link 2023-12-12T09:52:55.673 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:55.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:55.674 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:52:55.760 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-12-12T09:52:55.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:55.764 DEBUG:teuthology.orchestra.run.smithi072:> route 2023-12-12T09:52:55.817 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2023-12-12T09:52:55.817 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:52:55.817 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-12T09:52:55.817 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-12T09:52:55.817 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:52:55.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:55.818 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:55.818 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:55.818 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-12T09:52:55.818 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-12T09:52:55.818 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-12T09:52:55.818 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:56.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:56.001 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:52:56.051 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-12T09:52:56.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:56.051 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2023-12-12T09:52:56.102 INFO:teuthology.orchestra.run.smithi072.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-12T09:52:56.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:56.103 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:56.103 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:56.103 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-12T09:52:56.103 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-12T09:52:56.103 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:56.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:56.243 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-12T09:52:56.332 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether b6:8c:07:10:87:56 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::b48c:7ff:fe10:8756/64 scope link 2023-12-12T09:52:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:56.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:56.336 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-12T09:52:56.425 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether b6:8c:07:10:87:56 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::b48c:7ff:fe10:8756/64 scope link 2023-12-12T09:52:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:56.429 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-12T09:52:56.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:56.430 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:56.430 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:56.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-12T09:52:56.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-12T09:52:56.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-12T09:52:56.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-12T09:52:56.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-12T09:52:56.430 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:56.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:56.698 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:52:56.698 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:52:56.698 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.1 up 2023-12-12T09:52:56.698 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.1 master ceph-brx 2023-12-12T09:52:56.698 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:52:56.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:52:56.868 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-12T09:52:56.915 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-12T09:52:56.916 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:52:56.916 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-12T09:52:56.963 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-12-12T09:52:56.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:56.964 DEBUG:teuthology.orchestra.run.smithi072:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:52:57.100 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:52:57.100 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:52:57.135 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:52:57.135 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:52:57.135 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.72:3300,172.21.15.113:3300,172.21.15.113:3301" 2023-12-12T09:52:57.135 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with new device syntax: 2@bd07f350-42f2-4d45-9a3d-13364a8566fe.cephfs=/ 2023-12-12T09:52:57.135 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.72:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2023-12-12T09:52:57.137 INFO:teuthology.orchestra.run.smithi072.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-12-12T09:52:57.137 INFO:teuthology.orchestra.run.smithi072.stdout: You just mounted a file system that supports labels which does not 2023-12-12T09:52:57.137 INFO:teuthology.orchestra.run.smithi072.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-12T09:52:57.137 INFO:teuthology.orchestra.run.smithi072.stdout: applications will generate AVC messages and not be allowed access to 2023-12-12T09:52:57.137 INFO:teuthology.orchestra.run.smithi072.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-12T09:52:57.141 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-12T09:52:57.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:57.141 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-12T09:52:57.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:57.233 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:52:57.287 INFO:teuthology.orchestra.run.smithi072.stdout:client4588 2023-12-12T09:52:57.287 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:52:57.287 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:52:57.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:57.336 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:52:57.440 INFO:teuthology.orchestra.run.smithi072.stdout:client4588 2023-12-12T09:52:57.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:57.441 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:52:57.503 INFO:teuthology.orchestra.run.smithi072.stdout:client4588 2023-12-12T09:52:57.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:57.504 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/bd07f350-42f2-4d45-9a3d-13364a8566fe.client4588/status') 2023-12-12T09:52:57.594 INFO:teuthology.orchestra.run.smithi072.stdout:instance: client.4588 (3)172.21.15.72:0/2690623475 2023-12-12T09:52:57.594 INFO:teuthology.orchestra.run.smithi072.stdout:blocklisted: false 2023-12-12T09:52:57.595 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records in 2023-12-12T09:52:57.595 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records out 2023-12-12T09:52:57.595 INFO:teuthology.orchestra.run.smithi072.stderr:70 bytes copied, 6.3546e-05 s, 1.1 MB/s 2023-12-12T09:52:57.598 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-12-12T09:52:57.598 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-12T09:52:57.598 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-12T09:52:57.598 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-12T09:52:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.811+0000 7fa1c8ea3640 1 Processor -- start 2023-12-12T09:52:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.811+0000 7fa1c8ea3640 1 -- start start 2023-12-12T09:52:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.812+0000 7fa1c8ea3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1c40d13c0 0x7fa1c40ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.812+0000 7fa1c8ea3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d0d70 0x7fa1c40d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.812+0000 7fa1c8ea3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa1c40cf6a0 0x7fa1c40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.812+0000 7fa1c8ea3640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c40b2280 con 0x7fa1c40cf6a0 2023-12-12T09:52:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.812+0000 7fa1c8ea3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c40cee70 con 0x7fa1c40d13c0 2023-12-12T09:52:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.812+0000 7fa1c8ea3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c40d1190 con 0x7fa1c40d0d70 2023-12-12T09:52:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c3fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa1c40cf6a0 0x7fa1c40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c3fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa1c40cf6a0 0x7fa1c40cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60722/0 (socket says 172.21.15.72:60722) 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c3fff640 1 -- 172.21.15.72:0/3254262305 learned_addr learned my addr 172.21.15.72:0/3254262305 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1bbfff640 1 --2- 172.21.15.72:0/3254262305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d0d70 0x7fa1c40d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c37fe640 1 --2- 172.21.15.72:0/3254262305 >> v2:172.21.15.113:3300/0 conn(0x7fa1c40d13c0 0x7fa1c40ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c3fff640 1 -- 172.21.15.72:0/3254262305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d0d70 msgr2=0x7fa1c40d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c3fff640 1 --2- 172.21.15.72:0/3254262305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d0d70 0x7fa1c40d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c3fff640 1 -- 172.21.15.72:0/3254262305 >> v2:172.21.15.113:3300/0 conn(0x7fa1c40d13c0 msgr2=0x7fa1c40ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c3fff640 1 --2- 172.21.15.72:0/3254262305 >> v2:172.21.15.113:3300/0 conn(0x7fa1c40d13c0 0x7fa1c40ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c3fff640 1 -- 172.21.15.72:0/3254262305 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c40d1ce0 con 0x7fa1c40cf6a0 2023-12-12T09:52:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.813+0000 7fa1c37fe640 1 --2- 172.21.15.72:0/3254262305 >> v2:172.21.15.113:3300/0 conn(0x7fa1c40d13c0 0x7fa1c40ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.814+0000 7fa1c3fff640 1 --2- 172.21.15.72:0/3254262305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40cf6a0 0x7fa1c40cfb10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa1c408b740 tx=0x7fa1bc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ef70fcca097cd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.815+0000 7fa1c2ffd640 1 -- 172.21.15.72:0/3254262305 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc02c9f0 con 0x7fa1c40cf6a0 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.815+0000 7fa1c2ffd640 1 -- 172.21.15.72:0/3254262305 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc02cb50 con 0x7fa1c40cf6a0 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.815+0000 7fa1c2ffd640 1 -- 172.21.15.72:0/3254262305 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0343f0 con 0x7fa1c40cf6a0 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.815+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/3254262305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40cf6a0 msgr2=0x7fa1c40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.815+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/3254262305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40cf6a0 0x7fa1c40cfb10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa1c408b740 tx=0x7fa1bc02be50 comp rx=0 tx=0).stop 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.816+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/3254262305 shutdown_connections 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.816+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/3254262305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40cf6a0 0x7fa1c40cfb10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.816+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/3254262305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d0d70 0x7fa1c40d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.816+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/3254262305 >> v2:172.21.15.113:3300/0 conn(0x7fa1c40d13c0 0x7fa1c40ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.816+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/3254262305 >> 172.21.15.72:0/3254262305 conn(0x7fa1c409c260 msgr2=0x7fa1c40d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.816+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/3254262305 shutdown_connections 2023-12-12T09:52:57.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.816+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/3254262305 wait complete. 2023-12-12T09:52:57.819 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.817+0000 7fa1c8ea3640 1 Processor -- start 2023-12-12T09:52:57.819 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.818+0000 7fa1c8ea3640 1 -- start start 2023-12-12T09:52:57.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.818+0000 7fa1c8ea3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c40cf6a0 0x7fa1c40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:57.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.818+0000 7fa1c8ea3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40d0d70 0x7fa1c40b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:57.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.818+0000 7fa1c8ea3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d13c0 0x7fa1c41560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1c8ea3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c40b4de0 con 0x7fa1c40d0d70 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1c8ea3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c4159fe0 con 0x7fa1c40cf6a0 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1c8ea3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c4159260 con 0x7fa1c40d13c0 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1bbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40d0d70 0x7fa1c40b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1bbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40d0d70 0x7fa1c40b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60734/0 (socket says 172.21.15.72:60734) 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1bbfff640 1 -- 172.21.15.72:0/306223998 learned_addr learned my addr 172.21.15.72:0/306223998 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1c37fe640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c40cf6a0 0x7fa1c40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1c3fff640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d13c0 0x7fa1c41560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1bbfff640 1 -- 172.21.15.72:0/306223998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d13c0 msgr2=0x7fa1c41560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1bbfff640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d13c0 0x7fa1c41560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1bbfff640 1 -- 172.21.15.72:0/306223998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c40cf6a0 msgr2=0x7fa1c40ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1bbfff640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c40cf6a0 0x7fa1c40ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.819+0000 7fa1bbfff640 1 -- 172.21.15.72:0/306223998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1bc009b30 con 0x7fa1c40d0d70 2023-12-12T09:52:57.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.820+0000 7fa1bbfff640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40d0d70 0x7fa1c40b3c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00d830 tx=0x7fa1ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:57.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.820+0000 7fa1c0ff9640 1 -- 172.21.15.72:0/306223998 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0096a0 con 0x7fa1c40d0d70 2023-12-12T09:52:57.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.820+0000 7fa1c0ff9640 1 -- 172.21.15.72:0/306223998 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac009800 con 0x7fa1c40d0d70 2023-12-12T09:52:57.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.820+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c4157110 con 0x7fa1c40d0d70 2023-12-12T09:52:57.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.820+0000 7fa1c0ff9640 1 -- 172.21.15.72:0/306223998 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac009b90 con 0x7fa1c40d0d70 2023-12-12T09:52:57.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.820+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c41597e0 con 0x7fa1c40d0d70 2023-12-12T09:52:57.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.821+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c4074450 con 0x7fa1c40d0d70 2023-12-12T09:52:57.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.822+0000 7fa1c0ff9640 1 -- 172.21.15.72:0/306223998 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1ac014410 con 0x7fa1c40d0d70 2023-12-12T09:52:57.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.823+0000 7fa1c0ff9640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa1a406ae40 0x7fa1a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:57.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.823+0000 7fa1c0ff9640 1 -- 172.21.15.72:0/306223998 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa1ac08a070 con 0x7fa1c40d0d70 2023-12-12T09:52:57.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.823+0000 7fa1c37fe640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa1a406ae40 0x7fa1a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:57.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.824+0000 7fa1c37fe640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa1a406ae40 0x7fa1a406d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4008850 tx=0x7fa1b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:57.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:57.827+0000 7fa1c0ff9640 1 -- 172.21.15.72:0/306223998 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1ac0bce90 con 0x7fa1c40d0d70 2023-12-12T09:52:58.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.057+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa1c40d2d70 con 0x7fa1c40d0d70 2023-12-12T09:52:58.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.058+0000 7fa1c0ff9640 1 -- 172.21.15.72:0/306223998 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa1ac054d50 con 0x7fa1c40d0d70 2023-12-12T09:52:58.060 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-12T09:52:58.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa1a406ae40 msgr2=0x7fa1a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:58.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa1a406ae40 0x7fa1a406d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4008850 tx=0x7fa1b4008040 comp rx=0 tx=0).stop 2023-12-12T09:52:58.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40d0d70 msgr2=0x7fa1c40b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:58.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40d0d70 0x7fa1c40b3c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00d830 tx=0x7fa1ac00dcf0 comp rx=0 tx=0).stop 2023-12-12T09:52:58.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 shutdown_connections 2023-12-12T09:52:58.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa1a406ae40 0x7fa1a406d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:58.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1c40d13c0 0x7fa1c41560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:58.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c40d0d70 0x7fa1c40b3c70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:58.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 --2- 172.21.15.72:0/306223998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c40cf6a0 0x7fa1c40ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:58.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 >> 172.21.15.72:0/306223998 conn(0x7fa1c409c260 msgr2=0x7fa1c40b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:58.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 shutdown_connections 2023-12-12T09:52:58.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:52:58.061+0000 7fa1c8ea3640 1 -- 172.21.15.72:0/306223998 wait complete. 2023-12-12T09:52:58.082 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-12T09:52:58.083 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:52:58.083 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2023-12-12T09:52:58.083 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-12T09:52:58.083 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-12T09:52:58.083 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-12T09:52:58.083 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-12T09:52:58.083 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:52:58.083 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-12T09:52:58.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:58.083 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2023-12-12T09:52:58.142 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:52:58.142 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:52:58.142 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:52:58.142 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:58.142 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2023-12-12T09:52:58.142 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42319sec preferred_lft 42319sec 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link 2023-12-12T09:52:58.143 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:52:58.144 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-12T09:52:58.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:58.145 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:52:58.145 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:52:58.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add name ceph-brx type bridge 2023-12-12T09:52:58.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr flush dev ceph-brx 2023-12-12T09:52:58.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set ceph-brx up 2023-12-12T09:52:58.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-12T09:52:58.145 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:52:58.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:58.634 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:52:58.707 INFO:teuthology.orchestra.run.smithi113.stdout:1 2023-12-12T09:52:58.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:58.711 DEBUG:teuthology.orchestra.run.smithi113:> route 2023-12-12T09:52:58.763 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2023-12-12T09:52:58.763 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:52:58.763 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-12T09:52:58.763 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-12T09:52:58.763 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:52:58.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:58.763 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:52:58.763 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:52:58.763 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-12T09:52:58.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-12T09:52:58.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-12T09:52:58.764 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:52:59.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:59.524 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-12-12T09:52:59.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:59.573 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2023-12-12T09:52:59.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:59.619 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:52:59.619 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:52:59.619 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-12T09:52:59.619 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-12T09:52:59.619 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:52:59.789 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-12T09:52:59.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:52:59.790 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:52:59.790 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:52:59.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-12T09:52:59.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-12T09:52:59.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-12T09:52:59.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-12T09:52:59.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-12T09:52:59.790 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:53:00.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:00.046 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:53:00.046 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:53:00.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2023-12-12T09:53:00.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.0 master ceph-brx 2023-12-12T09:53:00.046 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:53:00.199 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:53:00.200 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-12T09:53:00.243 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-12T09:53:00.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:53:00.244 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-12T09:53:00.334 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-12-12T09:53:00.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:00.335 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:00.513 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:00.513 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:53:01.007 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:01.007 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:53:01.007 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.72:3300,172.21.15.113:3300,172.21.15.113:3301" 2023-12-12T09:53:01.007 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 1@bd07f350-42f2-4d45-9a3d-13364a8566fe.cephfs=/ 2023-12-12T09:53:01.007 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.72:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2023-12-12T09:53:01.012 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-12-12T09:53:01.012 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2023-12-12T09:53:01.012 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-12T09:53:01.012 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2023-12-12T09:53:01.012 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-12T09:53:01.018 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-12T09:53:01.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:01.018 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-12T09:53:01.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:01.110 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:53:01.166 INFO:teuthology.orchestra.run.smithi113.stdout:client4854 2023-12-12T09:53:01.166 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:53:01.166 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:01.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:01.220 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:53:01.276 INFO:teuthology.orchestra.run.smithi113.stdout:client4854 2023-12-12T09:53:01.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:01.276 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:53:01.331 INFO:teuthology.orchestra.run.smithi113.stdout:client4854 2023-12-12T09:53:01.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:01.332 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/bd07f350-42f2-4d45-9a3d-13364a8566fe.client4854/status') 2023-12-12T09:53:01.424 INFO:teuthology.orchestra.run.smithi113.stdout:instance: client.4854 (3)172.21.15.113:0/1631010637 2023-12-12T09:53:01.425 INFO:teuthology.orchestra.run.smithi113.stdout:blocklisted: false 2023-12-12T09:53:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records in 2023-12-12T09:53:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records out 2023-12-12T09:53:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:71 bytes copied, 6.5824e-05 s, 1.1 MB/s 2023-12-12T09:53:01.428 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-12-12T09:53:01.429 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-12T09:53:01.429 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-12T09:53:01.429 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-12T09:53:01.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.644+0000 7f962e600640 1 Processor -- start 2023-12-12T09:53:01.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.644+0000 7f962e600640 1 -- start start 2023-12-12T09:53:01.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.646+0000 7f962e600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 0x7f96280ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:01.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.646+0000 7f962e600640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f962806d520 0x7f96281626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:01.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.646+0000 7f962e600640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96280cde40 con 0x7f962815f260 2023-12-12T09:53:01.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.646+0000 7f962e600640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280cd040 con 0x7f962806d520 2023-12-12T09:53:01.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.646+0000 7f962e600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9628162bf0 con 0x7f96280708d0 2023-12-12T09:53:01.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.646+0000 7f962cb9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f962806d520 0x7f96281626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:01.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.646+0000 7f9627fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f962815f260 0x7f96280701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:45084/0 (socket says 172.21.15.72:45084) 2023-12-12T09:53:01.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.646+0000 7f9627fff640 1 -- 172.21.15.72:0/3834280379 learned_addr learned my addr 172.21.15.72:0/3834280379 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:01.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.647+0000 7f96277fe640 1 --2- 172.21.15.72:0/3834280379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 0x7f96280ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:01.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.647+0000 7f9626ffd640 1 -- 172.21.15.72:0/3834280379 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552722669 0 0) 0x7f96280cde40 con 0x7f962815f260 2023-12-12T09:53:01.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.647+0000 7f9626ffd640 1 -- 172.21.15.72:0/3834280379 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9614003520 con 0x7f962815f260 2023-12-12T09:53:01.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.647+0000 7f962cb9d640 1 -- 172.21.15.72:0/3834280379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 msgr2=0x7f96280ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:01.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.647+0000 7f962cb9d640 1 --2- 172.21.15.72:0/3834280379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 0x7f96280ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.647+0000 7f962cb9d640 1 -- 172.21.15.72:0/3834280379 >> v1:172.21.15.72:6789/0 conn(0x7f962815f260 legacy=0x7f96280701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:01.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.647+0000 7f962cb9d640 1 -- 172.21.15.72:0/3834280379 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96280cd230 con 0x7f962806d520 2023-12-12T09:53:01.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.648+0000 7f962cb9d640 1 --2- 172.21.15.72:0/3834280379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962806d520 0x7f96281626b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9620009d90 tx=0x7f962002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de0e5493857440c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:01.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.648+0000 7f9626ffd640 1 -- 172.21.15.72:0/3834280379 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962002cc50 con 0x7f962806d520 2023-12-12T09:53:01.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.648+0000 7f9626ffd640 1 -- 172.21.15.72:0/3834280379 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962002cdb0 con 0x7f962806d520 2023-12-12T09:53:01.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.648+0000 7f9626ffd640 1 -- 172.21.15.72:0/3834280379 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620034660 con 0x7f962806d520 2023-12-12T09:53:01.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.649+0000 7f962e600640 1 -- 172.21.15.72:0/3834280379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962806d520 msgr2=0x7f96281626b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:01.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.649+0000 7f962e600640 1 --2- 172.21.15.72:0/3834280379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962806d520 0x7f96281626b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9620009d90 tx=0x7f962002bea0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.649+0000 7f962e600640 1 -- 172.21.15.72:0/3834280379 shutdown_connections 2023-12-12T09:53:01.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.649+0000 7f962e600640 1 --2- 172.21.15.72:0/3834280379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962806d520 0x7f96281626b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.649+0000 7f962e600640 1 --2- 172.21.15.72:0/3834280379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 0x7f96280ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.649+0000 7f962e600640 1 -- 172.21.15.72:0/3834280379 >> 172.21.15.72:0/3834280379 conn(0x7f96280b0a90 msgr2=0x7f96280b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:01.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.649+0000 7f962e600640 1 -- 172.21.15.72:0/3834280379 shutdown_connections 2023-12-12T09:53:01.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.649+0000 7f962e600640 1 -- 172.21.15.72:0/3834280379 wait complete. 2023-12-12T09:53:01.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.651+0000 7f962e600640 1 Processor -- start 2023-12-12T09:53:01.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.651+0000 7f962e600640 1 -- start start 2023-12-12T09:53:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806d520 0x7f96281fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962e600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 0x7f96280ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962e600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962815f260 0x7f96280b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962e600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280abe60 con 0x7f962815f260 2023-12-12T09:53:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96281fa8c0 con 0x7f962806d520 2023-12-12T09:53:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962e600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96281faba0 con 0x7f96280708d0 2023-12-12T09:53:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962cb9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962815f260 0x7f96280b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:01.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962cb9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962815f260 0x7f96280b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60756/0 (socket says 172.21.15.72:60756) 2023-12-12T09:53:01.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.652+0000 7f962cb9d640 1 -- 172.21.15.72:0/151835597 learned_addr learned my addr 172.21.15.72:0/151835597 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f9627fff640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806d520 0x7f96281fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f962cb9d640 1 -- 172.21.15.72:0/151835597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 msgr2=0x7f96280ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f962cb9d640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 0x7f96280ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f962cb9d640 1 -- 172.21.15.72:0/151835597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806d520 msgr2=0x7f96281fa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f962cb9d640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806d520 0x7f96281fa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f962cb9d640 1 -- 172.21.15.72:0/151835597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9620009a70 con 0x7f962815f260 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f9627fff640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806d520 0x7f96281fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f962cb9d640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962815f260 0x7f96280b3f40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9620009d90 tx=0x7f9620004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f9624ff9640 1 -- 172.21.15.72:0/151835597 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620040650 con 0x7f962815f260 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.653+0000 7f9624ff9640 1 -- 172.21.15.72:0/151835597 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96200407b0 con 0x7f962815f260 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.654+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96281fae80 con 0x7f962815f260 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.654+0000 7f9624ff9640 1 -- 172.21.15.72:0/151835597 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620040b60 con 0x7f962815f260 2023-12-12T09:53:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.654+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96280a84f0 con 0x7f962815f260 2023-12-12T09:53:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.655+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96280745d0 con 0x7f962815f260 2023-12-12T09:53:01.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.656+0000 7f9624ff9640 1 -- 172.21.15.72:0/151835597 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9620040cc0 con 0x7f962815f260 2023-12-12T09:53:01.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.657+0000 7f9624ff9640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f960806ae40 0x7f960806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:01.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.657+0000 7f9624ff9640 1 -- 172.21.15.72:0/151835597 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f96200ac6f0 con 0x7f962815f260 2023-12-12T09:53:01.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.657+0000 7f9627fff640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f960806ae40 0x7f960806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:01.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.658+0000 7f9627fff640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f960806ae40 0x7f960806d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96180011e0 tx=0x7f9618009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:01.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.661+0000 7f9624ff9640 1 -- 172.21.15.72:0/151835597 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9620077470 con 0x7f962815f260 2023-12-12T09:53:01.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.884+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96280a86e0 con 0x7f962815f260 2023-12-12T09:53:01.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.885+0000 7f9624ff9640 1 -- 172.21.15.72:0/151835597 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9620009450 con 0x7f962815f260 2023-12-12T09:53:01.887 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-12T09:53:01.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f960806ae40 msgr2=0x7f960806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:01.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f960806ae40 0x7f960806d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96180011e0 tx=0x7f9618009040 comp rx=0 tx=0).stop 2023-12-12T09:53:01.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962815f260 msgr2=0x7f96280b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:01.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962815f260 0x7f96280b3f40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9620009d90 tx=0x7f9620004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:01.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 shutdown_connections 2023-12-12T09:53:01.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f960806ae40 0x7f960806d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962815f260 0x7f96280b3f40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96280708d0 0x7f96280ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 --2- 172.21.15.72:0/151835597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806d520 0x7f96281fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:01.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 >> 172.21.15.72:0/151835597 conn(0x7f96280b0a90 msgr2=0x7f96280aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:01.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 shutdown_connections 2023-12-12T09:53:01.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:01.887+0000 7f962e600640 1 -- 172.21.15.72:0/151835597 wait complete. 2023-12-12T09:53:01.907 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-12T09:53:01.907 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:53:01.907 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2023-12-12T09:53:01.907 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-12T09:53:01.907 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-12T09:53:01.907 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-12T09:53:01.907 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-12T09:53:01.908 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:53:01.908 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-12T09:53:01.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:01.908 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2023-12-12T09:53:01.969 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2023-12-12T09:53:01.970 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42316sec preferred_lft 42316sec 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether f2:93:b2:71:79:14 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::fc40:30ff:fe16:f01c/64 scope link 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether f2:93:b2:71:79:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-12T09:53:01.971 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f093:b2ff:fe71:7914/64 scope link 2023-12-12T09:53:01.972 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:01.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:01.973 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:53:02.065 INFO:teuthology.orchestra.run.smithi113.stdout:1 2023-12-12T09:53:02.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:02.070 DEBUG:teuthology.orchestra.run.smithi113:> route 2023-12-12T09:53:02.121 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2023-12-12T09:53:02.121 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:53:02.121 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-12T09:53:02.121 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-12T09:53:02.121 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:53:02.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:02.122 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:53:02.122 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:53:02.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-12T09:53:02.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-12T09:53:02.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-12T09:53:02.122 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:53:02.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:02.303 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-12-12T09:53:02.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-12T09:53:02.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:02.357 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2023-12-12T09:53:02.408 INFO:teuthology.orchestra.run.smithi113.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-12T09:53:02.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:02.409 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:53:02.409 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:53:02.409 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-12T09:53:02.409 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-12T09:53:02.409 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:53:02.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:02.552 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-12T09:53:02.639 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:53:02.639 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:53:02.639 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:53:02.639 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:02.639 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2023-12-12T09:53:02.639 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:02.639 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-12T09:53:02.640 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ce:00:ef:93:82:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-12T09:53:02.640 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-12T09:53:02.640 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:02.640 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::cc00:efff:fe93:82db/64 scope link 2023-12-12T09:53:02.640 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:02.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:02.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-12T09:53:02.785 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:53:02.785 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ce:00:ef:93:82:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::cc00:efff:fe93:82db/64 scope link 2023-12-12T09:53:02.786 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:02.789 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-12T09:53:02.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:02.790 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:53:02.790 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:53:02.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-12T09:53:02.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-12T09:53:02.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-12T09:53:02.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-12T09:53:02.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-12T09:53:02.790 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:53:02.985 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:53:02.995 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:53:03.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:03.058 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-12-12T09:53:03.058 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-12-12T09:53:03.058 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.1 up 2023-12-12T09:53:03.058 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.1 master ceph-brx 2023-12-12T09:53:03.059 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-12-12T09:53:03.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:53:03.230 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-12T09:53:03.285 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-12T09:53:03.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:53:03.286 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-12T09:53:03.338 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-12-12T09:53:03.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:03.338 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:03.475 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:03.475 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:53:03.510 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:03.510 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:53:03.511 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.72:3300,172.21.15.113:3300,172.21.15.113:3301" 2023-12-12T09:53:03.511 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 3@bd07f350-42f2-4d45-9a3d-13364a8566fe.cephfs=/ 2023-12-12T09:53:03.511 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.72:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2023-12-12T09:53:03.513 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-12-12T09:53:03.513 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2023-12-12T09:53:03.513 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-12T09:53:03.513 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2023-12-12T09:53:03.513 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-12T09:53:03.517 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-12T09:53:03.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:03.517 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-12T09:53:03.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:03.609 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:53:03.666 INFO:teuthology.orchestra.run.smithi113.stdout:client4866 2023-12-12T09:53:03.667 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:53:03.667 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:03.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:03.720 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:53:03.780 INFO:teuthology.orchestra.run.smithi113.stdout:client4866 2023-12-12T09:53:03.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:03.780 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:53:03.837 INFO:teuthology.orchestra.run.smithi113.stdout:client4866 2023-12-12T09:53:03.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:03.837 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/bd07f350-42f2-4d45-9a3d-13364a8566fe.client4866/status') 2023-12-12T09:53:03.927 INFO:teuthology.orchestra.run.smithi113.stdout:instance: client.4866 (3)172.21.15.113:0/1802417228 2023-12-12T09:53:03.927 INFO:teuthology.orchestra.run.smithi113.stdout:blocklisted: false 2023-12-12T09:53:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records in 2023-12-12T09:53:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records out 2023-12-12T09:53:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:71 bytes copied, 5.789e-05 s, 1.2 MB/s 2023-12-12T09:53:03.934 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-12T09:53:03.965 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:04.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.175+0000 7f450b423640 1 Processor -- start 2023-12-12T09:53:04.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.175+0000 7f450b423640 1 -- start start 2023-12-12T09:53:04.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.176+0000 7f450b423640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 0x7f45040b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.176+0000 7f450b423640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f450406c410 0x7f450415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.176+0000 7f450b423640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45040b3bb0 con 0x7f4504143e00 2023-12-12T09:53:04.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.176+0000 7f450b423640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45041441e0 con 0x7f450406c410 2023-12-12T09:53:04.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.176+0000 7f450b423640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4504072680 con 0x7f4504072180 2023-12-12T09:53:04.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.177+0000 7f45099c0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f450406c410 0x7f450415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.177+0000 7f45089be640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 0x7f45040b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.177+0000 7f45099c0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f450406c410 0x7f450415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:54920/0 (socket says 172.21.15.72:54920) 2023-12-12T09:53:04.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.177+0000 7f45099c0640 1 -- 172.21.15.72:0/3995900917 learned_addr learned my addr 172.21.15.72:0/3995900917 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:04.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.178+0000 7f45099c0640 1 -- 172.21.15.72:0/3995900917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 msgr2=0x7f45040b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:04.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.178+0000 7f45099c0640 1 --2- 172.21.15.72:0/3995900917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 0x7f45040b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.178+0000 7f45099c0640 1 -- 172.21.15.72:0/3995900917 >> v1:172.21.15.113:6789/0 conn(0x7f4504143e00 legacy=0x7f45040b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.178+0000 7f45099c0640 1 -- 172.21.15.72:0/3995900917 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040b1e30 con 0x7f450406c410 2023-12-12T09:53:04.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.179+0000 7f45099c0640 1 --2- 172.21.15.72:0/3995900917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 0x7f450415ecb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009e50 tx=0x7f44fc02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d818e80b35f463fa server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:04.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f44fbfff640 1 -- 172.21.15.72:0/3995900917 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc02cb00 con 0x7f450406c410 2023-12-12T09:53:04.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f44fbfff640 1 -- 172.21.15.72:0/3995900917 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc02cc60 con 0x7f450406c410 2023-12-12T09:53:04.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f44fbfff640 1 -- 172.21.15.72:0/3995900917 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0343f0 con 0x7f450406c410 2023-12-12T09:53:04.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f450b423640 1 -- 172.21.15.72:0/3995900917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 msgr2=0x7f450415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f450b423640 1 --2- 172.21.15.72:0/3995900917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 0x7f450415ecb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009e50 tx=0x7f44fc02bf60 comp rx=0 tx=0).stop 2023-12-12T09:53:04.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f450b423640 1 -- 172.21.15.72:0/3995900917 shutdown_connections 2023-12-12T09:53:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f450b423640 1 --2- 172.21.15.72:0/3995900917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 0x7f450415ecb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f450b423640 1 --2- 172.21.15.72:0/3995900917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 0x7f45040b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f450b423640 1 -- 172.21.15.72:0/3995900917 >> 172.21.15.72:0/3995900917 conn(0x7f45040b0a90 msgr2=0x7f45040b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f450b423640 1 -- 172.21.15.72:0/3995900917 shutdown_connections 2023-12-12T09:53:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.180+0000 7f450b423640 1 -- 172.21.15.72:0/3995900917 wait complete. 2023-12-12T09:53:04.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.182+0000 7f450b423640 1 Processor -- start 2023-12-12T09:53:04.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.182+0000 7f450b423640 1 -- start start 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.183+0000 7f450b423640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 0x7f450413ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.183+0000 7f450b423640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 0x7f450413b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.183+0000 7f450b423640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4504143e00 0x7f45040cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.183+0000 7f450b423640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040ce2c0 con 0x7f4504143e00 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.183+0000 7f450b423640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040cd740 con 0x7f450406c410 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.183+0000 7f450b423640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040cda20 con 0x7f4504072180 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45099c0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4504143e00 0x7f45040cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45099c0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4504143e00 0x7f45040cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60768/0 (socket says 172.21.15.72:60768) 2023-12-12T09:53:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45099c0640 1 -- 172.21.15.72:0/3301874619 learned_addr learned my addr 172.21.15.72:0/3301874619 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45089be640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 0x7f450413b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45091bf640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 0x7f450413ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45099c0640 1 -- 172.21.15.72:0/3301874619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 msgr2=0x7f450413b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45099c0640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 0x7f450413b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45099c0640 1 -- 172.21.15.72:0/3301874619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 msgr2=0x7f450413ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45099c0640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 0x7f450413ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.184+0000 7f45099c0640 1 -- 172.21.15.72:0/3301874619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc009b30 con 0x7f4504143e00 2023-12-12T09:53:04.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.185+0000 7f45099c0640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4504143e00 0x7f45040cc980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009e50 tx=0x7f44fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:04.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.185+0000 7f44f9ffb640 1 -- 172.21.15.72:0/3301874619 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc009510 con 0x7f4504143e00 2023-12-12T09:53:04.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.185+0000 7f44f9ffb640 1 -- 172.21.15.72:0/3301874619 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc03e450 con 0x7f4504143e00 2023-12-12T09:53:04.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.185+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45040cdd00 con 0x7f4504143e00 2023-12-12T09:53:04.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.185+0000 7f44f9ffb640 1 -- 172.21.15.72:0/3301874619 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc03e800 con 0x7f4504143e00 2023-12-12T09:53:04.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.185+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4504137280 con 0x7f4504143e00 2023-12-12T09:53:04.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.186+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45040745d0 con 0x7f4504143e00 2023-12-12T09:53:04.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.187+0000 7f44f9ffb640 1 -- 172.21.15.72:0/3301874619 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44fc03e960 con 0x7f4504143e00 2023-12-12T09:53:04.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.189+0000 7f44f9ffb640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f44e806ae40 0x7f44e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.189+0000 7f44f9ffb640 1 -- 172.21.15.72:0/3301874619 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f44fc0ac750 con 0x7f4504143e00 2023-12-12T09:53:04.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.189+0000 7f45091bf640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f44e806ae40 0x7f44e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.190+0000 7f45091bf640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f44e806ae40 0x7f44e806d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4500000f60 tx=0x7f4500009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:04.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.194+0000 7f44f9ffb640 1 -- 172.21.15.72:0/3301874619 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44fc0773c0 con 0x7f4504143e00 2023-12-12T09:53:04.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.384+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f450406e020 con 0x7f4504143e00 2023-12-12T09:53:04.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.386+0000 7f44f9ffb640 1 -- 172.21.15.72:0/3301874619 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7f44fc07a9e0 con 0x7f4504143e00 2023-12-12T09:53:04.389 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:04.389 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:04.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f44e806ae40 msgr2=0x7f44e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f44e806ae40 0x7f44e806d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4500000f60 tx=0x7f4500009040 comp rx=0 tx=0).stop 2023-12-12T09:53:04.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4504143e00 msgr2=0x7f45040cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4504143e00 0x7f45040cc980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009e50 tx=0x7f44fc004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:04.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 shutdown_connections 2023-12-12T09:53:04.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f44e806ae40 0x7f44e806d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4504143e00 0x7f45040cc980 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4504072180 0x7f450413b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 --2- 172.21.15.72:0/3301874619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f450406c410 0x7f450413ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.388+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 >> 172.21.15.72:0/3301874619 conn(0x7f45040b0a90 msgr2=0x7f450406c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:04.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.389+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 shutdown_connections 2023-12-12T09:53:04.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.389+0000 7f450b423640 1 -- 172.21.15.72:0/3301874619 wait complete. 2023-12-12T09:53:04.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:04.408 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:04.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.609+0000 7fd248d6c640 1 Processor -- start 2023-12-12T09:53:04.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.609+0000 7fd248d6c640 1 -- start start 2023-12-12T09:53:04.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd248d6c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd24415d6f0 0x7fd2440cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd248d6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440d1890 0x7fd2440cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd248d6c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2440b2180 0x7fd2440cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd248d6c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2440ce6f0 con 0x7fd24415d6f0 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd248d6c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24415e780 con 0x7fd2440b2180 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd248d6c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440cb900 con 0x7fd2440d1890 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd2437fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd24415d6f0 0x7fd2440cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd2437fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd24415d6f0 0x7fd2440cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60772/0 (socket says 172.21.15.72:60772) 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.611+0000 7fd2437fe640 1 -- 172.21.15.72:0/2404841855 learned_addr learned my addr 172.21.15.72:0/2404841855 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.612+0000 7fd2437fe640 1 -- 172.21.15.72:0/2404841855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440d1890 msgr2=0x7fd2440cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.612+0000 7fd2437fe640 1 --2- 172.21.15.72:0/2404841855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440d1890 0x7fd2440cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.612+0000 7fd2437fe640 1 -- 172.21.15.72:0/2404841855 >> v2:172.21.15.113:3300/0 conn(0x7fd2440b2180 msgr2=0x7fd2440cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.612+0000 7fd2437fe640 1 --2- 172.21.15.72:0/2404841855 >> v2:172.21.15.113:3300/0 conn(0x7fd2440b2180 0x7fd2440cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.612+0000 7fd2437fe640 1 -- 172.21.15.72:0/2404841855 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24415e9d0 con 0x7fd24415d6f0 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.613+0000 7fd2437fe640 1 --2- 172.21.15.72:0/2404841855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd24415d6f0 0x7fd2440cb3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd24406ade0 tx=0x7fd23402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2041d61503cca99 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.613+0000 7fd242ffd640 1 -- 172.21.15.72:0/2404841855 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340317e0 con 0x7fd24415d6f0 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.613+0000 7fd242ffd640 1 -- 172.21.15.72:0/2404841855 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd234031940 con 0x7fd24415d6f0 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.613+0000 7fd242ffd640 1 -- 172.21.15.72:0/2404841855 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234031cd0 con 0x7fd24415d6f0 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 -- 172.21.15.72:0/2404841855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd24415d6f0 msgr2=0x7fd2440cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 --2- 172.21.15.72:0/2404841855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd24415d6f0 0x7fd2440cb3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd24406ade0 tx=0x7fd23402b880 comp rx=0 tx=0).stop 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 -- 172.21.15.72:0/2404841855 shutdown_connections 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 --2- 172.21.15.72:0/2404841855 >> v2:172.21.15.113:3300/0 conn(0x7fd2440b2180 0x7fd2440cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 --2- 172.21.15.72:0/2404841855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440d1890 0x7fd2440cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 --2- 172.21.15.72:0/2404841855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd24415d6f0 0x7fd2440cb3c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 -- 172.21.15.72:0/2404841855 >> 172.21.15.72:0/2404841855 conn(0x7fd2440d0d70 msgr2=0x7fd2440d1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 -- 172.21.15.72:0/2404841855 shutdown_connections 2023-12-12T09:53:04.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.614+0000 7fd248d6c640 1 -- 172.21.15.72:0/2404841855 wait complete. 2023-12-12T09:53:04.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.616+0000 7fd248d6c640 1 Processor -- start 2023-12-12T09:53:04.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.616+0000 7fd248d6c640 1 -- start start 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd248d6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440b2180 0x7fd244155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd248d6c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2440d1890 0x7fd2441530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd248d6c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24415d6f0 0x7fd24414b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd248d6c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2441535f0 con 0x7fd2440d1890 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd248d6c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440c5df0 con 0x7fd24415d6f0 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd248d6c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440c6040 con 0x7fd2440b2180 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2440d1890 0x7fd2441530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2440d1890 0x7fd2441530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60784/0 (socket says 172.21.15.72:60784) 2023-12-12T09:53:04.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd23bfff640 1 -- 172.21.15.72:0/936242641 learned_addr learned my addr 172.21.15.72:0/936242641 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.617+0000 7fd2437fe640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440b2180 0x7fd244155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd243fff640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24415d6f0 0x7fd24414b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd23bfff640 1 -- 172.21.15.72:0/936242641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440b2180 msgr2=0x7fd244155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd23bfff640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440b2180 0x7fd244155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd23bfff640 1 -- 172.21.15.72:0/936242641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24415d6f0 msgr2=0x7fd24414b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd23bfff640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24415d6f0 0x7fd24414b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd23bfff640 1 -- 172.21.15.72:0/936242641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2340094d0 con 0x7fd2440d1890 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd23bfff640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2440d1890 0x7fd2441530b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00b410 tx=0x7fd22c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:04.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd240ff9640 1 -- 172.21.15.72:0/936242641 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c011830 con 0x7fd2440d1890 2023-12-12T09:53:04.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd240ff9640 1 -- 172.21.15.72:0/936242641 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c0119b0 con 0x7fd2440d1890 2023-12-12T09:53:04.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd240ff9640 1 -- 172.21.15.72:0/936242641 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c011d60 con 0x7fd2440d1890 2023-12-12T09:53:04.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.618+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2440c6320 con 0x7fd2440d1890 2023-12-12T09:53:04.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.619+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24414b680 con 0x7fd2440d1890 2023-12-12T09:53:04.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.620+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2441f6a20 con 0x7fd2440d1890 2023-12-12T09:53:04.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.621+0000 7fd240ff9640 1 -- 172.21.15.72:0/936242641 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd22c01a400 con 0x7fd2440d1890 2023-12-12T09:53:04.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.621+0000 7fd240ff9640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd22406ae40 0x7fd22406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:04.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.622+0000 7fd240ff9640 1 -- 172.21.15.72:0/936242641 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd22c089b80 con 0x7fd2440d1890 2023-12-12T09:53:04.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.622+0000 7fd2437fe640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd22406ae40 0x7fd22406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:04.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.622+0000 7fd2437fe640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd22406ae40 0x7fd22406d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd234036000 tx=0x7fd23403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:04.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.630+0000 7fd240ff9640 1 -- 172.21.15.72:0/936242641 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2441f6a20 con 0x7fd2440d1890 2023-12-12T09:53:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.838+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2441f6e00 con 0x7fd2440d1890 2023-12-12T09:53:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.838+0000 7fd240ff9640 1 -- 172.21.15.72:0/936242641 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2441f6e00 con 0x7fd2440d1890 2023-12-12T09:53:04.841 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd22406ae40 msgr2=0x7fd22406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd22406ae40 0x7fd22406d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd234036000 tx=0x7fd23403d040 comp rx=0 tx=0).stop 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2440d1890 msgr2=0x7fd2441530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2440d1890 0x7fd2441530b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00b410 tx=0x7fd22c00b8d0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 shutdown_connections 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd22406ae40 0x7fd22406d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24415d6f0 0x7fd24414b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2440d1890 0x7fd2441530b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 --2- 172.21.15.72:0/936242641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2440b2180 0x7fd244155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 >> 172.21.15.72:0/936242641 conn(0x7fd2440d0d70 msgr2=0x7fd24406c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 shutdown_connections 2023-12-12T09:53:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:04.841+0000 7fd248d6c640 1 -- 172.21.15.72:0/936242641 wait complete. 2023-12-12T09:53:04.860 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:05.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.058+0000 7fe69c8d2640 1 Processor -- start 2023-12-12T09:53:05.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.058+0000 7fe69c8d2640 1 -- start start 2023-12-12T09:53:05.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.059+0000 7fe69c8d2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6980cb5b0 0x7fe69806c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69c8d2640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe698070c60 0x7fe698071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69c8d2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69806c770 con 0x7fe6980cb990 2023-12-12T09:53:05.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69c8d2640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69806ca50 con 0x7fe698070c60 2023-12-12T09:53:05.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69c8d2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69806cd30 con 0x7fe6980cb5b0 2023-12-12T09:53:05.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69759e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe698070c60 0x7fe698071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69759e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe698070c60 0x7fe698071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36048/0 (socket says 172.21.15.72:36048) 2023-12-12T09:53:05.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69759e640 1 -- 172.21.15.72:0/2757016886 learned_addr learned my addr 172.21.15.72:0/2757016886 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:05.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69759e640 1 -- 172.21.15.72:0/2757016886 >> v2:172.21.15.113:3300/0 conn(0x7fe6980cb5b0 msgr2=0x7fe69806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:05.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69759e640 1 --2- 172.21.15.72:0/2757016886 >> v2:172.21.15.113:3300/0 conn(0x7fe6980cb5b0 0x7fe69806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69759e640 1 -- 172.21.15.72:0/2757016886 >> v1:172.21.15.113:6789/0 conn(0x7fe6980cb990 legacy=0x7fe69815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:05.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.060+0000 7fe69759e640 1 -- 172.21.15.72:0/2757016886 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69806d010 con 0x7fe698070c60 2023-12-12T09:53:05.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.061+0000 7fe69759e640 1 --2- 172.21.15.72:0/2757016886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe698071110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6980cfba0 tx=0x7fe68802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7213d1071dee7e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe695d9b640 1 -- 172.21.15.72:0/2757016886 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688031870 con 0x7fe698070c60 2023-12-12T09:53:05.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe695d9b640 1 -- 172.21.15.72:0/2757016886 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6880319d0 con 0x7fe698070c60 2023-12-12T09:53:05.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe695d9b640 1 -- 172.21.15.72:0/2757016886 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688031d60 con 0x7fe698070c60 2023-12-12T09:53:05.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe69c8d2640 1 -- 172.21.15.72:0/2757016886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 msgr2=0x7fe698071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/2757016886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe698071110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6980cfba0 tx=0x7fe68802b910 comp rx=0 tx=0).stop 2023-12-12T09:53:05.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe69c8d2640 1 -- 172.21.15.72:0/2757016886 shutdown_connections 2023-12-12T09:53:05.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/2757016886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe698071110 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/2757016886 >> v2:172.21.15.113:3300/0 conn(0x7fe6980cb5b0 0x7fe69806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe69c8d2640 1 -- 172.21.15.72:0/2757016886 >> 172.21.15.72:0/2757016886 conn(0x7fe6980b0940 msgr2=0x7fe6980b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:05.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe69c8d2640 1 -- 172.21.15.72:0/2757016886 shutdown_connections 2023-12-12T09:53:05.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.062+0000 7fe69c8d2640 1 -- 172.21.15.72:0/2757016886 wait complete. 2023-12-12T09:53:05.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.064+0000 7fe69c8d2640 1 Processor -- start 2023-12-12T09:53:05.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.064+0000 7fe69c8d2640 1 -- start start 2023-12-12T09:53:05.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe69c8d2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe69814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe69c8d2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6980cb5b0 0x7fe69815de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe69c8d2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980cb990 0x7fe6980999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe69c8d2640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe698156c50 con 0x7fe698070c60 2023-12-12T09:53:05.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe69c8d2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69815d9c0 con 0x7fe6980cb990 2023-12-12T09:53:05.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe69c8d2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69815cdb0 con 0x7fe6980cb5b0 2023-12-12T09:53:05.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe696d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe69814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe696d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe69814d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36054/0 (socket says 172.21.15.72:36054) 2023-12-12T09:53:05.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.065+0000 7fe696d9d640 1 -- 172.21.15.72:0/903566438 learned_addr learned my addr 172.21.15.72:0/903566438 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe69759e640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980cb990 0x7fe6980999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe69659c640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6980cb5b0 0x7fe69815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe696d9d640 1 -- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6980cb5b0 msgr2=0x7fe69815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe696d9d640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6980cb5b0 0x7fe69815de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe696d9d640 1 -- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980cb990 msgr2=0x7fe6980999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe696d9d640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980cb990 0x7fe6980999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe696d9d640 1 -- 172.21.15.72:0/903566438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880094d0 con 0x7fe698070c60 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe69759e640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980cb990 0x7fe6980999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe696d9d640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe69814d340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe69000ba80 tx=0x7fe69000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe68f7fe640 1 -- 172.21.15.72:0/903566438 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69000cc00 con 0x7fe698070c60 2023-12-12T09:53:05.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe68f7fe640 1 -- 172.21.15.72:0/903566438 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69000cd80 con 0x7fe698070c60 2023-12-12T09:53:05.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69815d090 con 0x7fe698070c60 2023-12-12T09:53:05.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.066+0000 7fe68f7fe640 1 -- 172.21.15.72:0/903566438 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690014680 con 0x7fe698070c60 2023-12-12T09:53:05.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.067+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69815d650 con 0x7fe698070c60 2023-12-12T09:53:05.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.068+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe698067910 con 0x7fe698070c60 2023-12-12T09:53:05.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.069+0000 7fe68f7fe640 1 -- 172.21.15.72:0/903566438 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe69001d3b0 con 0x7fe698070c60 2023-12-12T09:53:05.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.069+0000 7fe68f7fe640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe67806ae70 0x7fe67806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.069+0000 7fe68f7fe640 1 -- 172.21.15.72:0/903566438 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe690089a00 con 0x7fe698070c60 2023-12-12T09:53:05.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.070+0000 7fe69659c640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe67806ae70 0x7fe67806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.070+0000 7fe69659c640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe67806ae70 0x7fe67806d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe6981582a0 tx=0x7fe680009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.074+0000 7fe68f7fe640 1 -- 172.21.15.72:0/903566438 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe690054650 con 0x7fe698070c60 2023-12-12T09:53:05.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.264+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6980cbdd0 con 0x7fe698070c60 2023-12-12T09:53:05.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.264+0000 7fe68f7fe640 1 -- 172.21.15.72:0/903566438 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe690019020 con 0x7fe698070c60 2023-12-12T09:53:05.267 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe67806ae70 msgr2=0x7fe67806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe67806ae70 0x7fe67806d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe6981582a0 tx=0x7fe680009040 comp rx=0 tx=0).stop 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 msgr2=0x7fe69814d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe69814d340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe69000ba80 tx=0x7fe69000bf40 comp rx=0 tx=0).stop 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 shutdown_connections 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe67806ae70 0x7fe67806d330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980cb990 0x7fe6980999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6980cb5b0 0x7fe69815de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 --2- 172.21.15.72:0/903566438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe698070c60 0x7fe69814d340 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 >> 172.21.15.72:0/903566438 conn(0x7fe6980b0940 msgr2=0x7fe6980b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:05.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 shutdown_connections 2023-12-12T09:53:05.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.267+0000 7fe69c8d2640 1 -- 172.21.15.72:0/903566438 wait complete. 2023-12-12T09:53:05.287 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.496+0000 7f6c98f05640 1 Processor -- start 2023-12-12T09:53:05.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.496+0000 7f6c98f05640 1 -- start start 2023-12-12T09:53:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.497+0000 7f6c98f05640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c940ce7e0 0x7f6c9415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.497+0000 7f6c98f05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9406a460 0x7f6c9406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.497+0000 7f6c98f05640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c94071cd0 con 0x7f6c9415af50 2023-12-12T09:53:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.497+0000 7f6c98f05640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9406ae30 con 0x7f6c940ce7e0 2023-12-12T09:53:05.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.497+0000 7f6c98f05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940717e0 con 0x7f6c9406a460 2023-12-12T09:53:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.498+0000 7f6c91d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c940ce7e0 0x7f6c9415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.498+0000 7f6c91d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c940ce7e0 0x7f6c9415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:52292/0 (socket says 172.21.15.72:52292) 2023-12-12T09:53:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.498+0000 7f6c91d9b640 1 -- 172.21.15.72:0/3254358165 learned_addr learned my addr 172.21.15.72:0/3254358165 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:05.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.498+0000 7f6c92d9d640 1 --2- 172.21.15.72:0/3254358165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9406a460 0x7f6c9406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.499+0000 7f6c91d9b640 1 -- 172.21.15.72:0/3254358165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9406a460 msgr2=0x7f6c9406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:05.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.499+0000 7f6c91d9b640 1 --2- 172.21.15.72:0/3254358165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9406a460 0x7f6c9406a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.499+0000 7f6c91d9b640 1 -- 172.21.15.72:0/3254358165 >> v1:172.21.15.113:6789/0 conn(0x7f6c9415af50 legacy=0x7f6c9406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.499+0000 7f6c91d9b640 1 -- 172.21.15.72:0/3254358165 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940719f0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.500+0000 7f6c91d9b640 1 --2- 172.21.15.72:0/3254358165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940ce7e0 0x7f6c9415ec00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009870 tx=0x7f6c8802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5ab3ce623ac1405 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.501+0000 7f6c9159a640 1 -- 172.21.15.72:0/3254358165 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880317c0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.501+0000 7f6c9159a640 1 -- 172.21.15.72:0/3254358165 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88031920 con 0x7f6c940ce7e0 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.501+0000 7f6c9159a640 1 -- 172.21.15.72:0/3254358165 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88031cb0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.501+0000 7f6c98f05640 1 -- 172.21.15.72:0/3254358165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940ce7e0 msgr2=0x7f6c9415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.501+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3254358165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940ce7e0 0x7f6c9415ec00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009870 tx=0x7f6c8802b860 comp rx=0 tx=0).stop 2023-12-12T09:53:05.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.502+0000 7f6c98f05640 1 -- 172.21.15.72:0/3254358165 shutdown_connections 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.502+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3254358165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9406a460 0x7f6c9406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.502+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3254358165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c940ce7e0 0x7f6c9415ec00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.502+0000 7f6c98f05640 1 -- 172.21.15.72:0/3254358165 >> 172.21.15.72:0/3254358165 conn(0x7f6c940cee70 msgr2=0x7f6c940cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.502+0000 7f6c98f05640 1 -- 172.21.15.72:0/3254358165 shutdown_connections 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.502+0000 7f6c98f05640 1 -- 172.21.15.72:0/3254358165 wait complete. 2023-12-12T09:53:05.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.503+0000 7f6c98f05640 1 Processor -- start 2023-12-12T09:53:05.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.504+0000 7f6c98f05640 1 -- start start 2023-12-12T09:53:05.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.504+0000 7f6c98f05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9406a460 0x7f6c94153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.504+0000 7f6c98f05640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c940ce7e0 0x7f6c94099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.504+0000 7f6c98f05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9415af50 0x7f6c9409a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c98f05640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c50e0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c98f05640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c4e00 con 0x7f6c9406a460 2023-12-12T09:53:05.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c98f05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c9e90 con 0x7f6c9415af50 2023-12-12T09:53:05.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c91d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c940ce7e0 0x7f6c94099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c91d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c940ce7e0 0x7f6c94099ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36066/0 (socket says 172.21.15.72:36066) 2023-12-12T09:53:05.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c91d9b640 1 -- 172.21.15.72:0/3154185918 learned_addr learned my addr 172.21.15.72:0/3154185918 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:05.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c92d9d640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9415af50 0x7f6c9409a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c9259c640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9406a460 0x7f6c94153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c91d9b640 1 -- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9415af50 msgr2=0x7f6c9409a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c91d9b640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9415af50 0x7f6c9409a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c91d9b640 1 -- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9406a460 msgr2=0x7f6c94153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c91d9b640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9406a460 0x7f6c94153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c91d9b640 1 -- 172.21.15.72:0/3154185918 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c880094d0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.505+0000 7f6c9259c640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9406a460 0x7f6c94153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.506+0000 7f6c91d9b640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c940ce7e0 0x7f6c94099ec0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6c8802b8a0 tx=0x7f6c880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.506+0000 7f6c82ffd640 1 -- 172.21.15.72:0/3154185918 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88032860 con 0x7f6c940ce7e0 2023-12-12T09:53:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.506+0000 7f6c82ffd640 1 -- 172.21.15.72:0/3154185918 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88032a00 con 0x7f6c940ce7e0 2023-12-12T09:53:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.506+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c940ca170 con 0x7f6c940ce7e0 2023-12-12T09:53:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.506+0000 7f6c82ffd640 1 -- 172.21.15.72:0/3154185918 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88032db0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.506+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941545d0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.507+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c940c9cf0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.508+0000 7f6c82ffd640 1 -- 172.21.15.72:0/3154185918 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c8804e450 con 0x7f6c940ce7e0 2023-12-12T09:53:05.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.509+0000 7f6c82ffd640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7406ae40 0x7f6c7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.509+0000 7f6c82ffd640 1 -- 172.21.15.72:0/3154185918 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6c880abb00 con 0x7f6c940ce7e0 2023-12-12T09:53:05.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.509+0000 7f6c9259c640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7406ae40 0x7f6c7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.510+0000 7f6c9259c640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7406ae40 0x7f6c7406d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c4890 tx=0x7f6c84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.513+0000 7f6c82ffd640 1 -- 172.21.15.72:0/3154185918 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c940c9cf0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.703+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c940c9cf0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.704+0000 7f6c82ffd640 1 -- 172.21.15.72:0/3154185918 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c940c9cf0 con 0x7f6c940ce7e0 2023-12-12T09:53:05.706 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:05.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7406ae40 msgr2=0x7f6c7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7406ae40 0x7f6c7406d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c4890 tx=0x7f6c84009040 comp rx=0 tx=0).stop 2023-12-12T09:53:05.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c940ce7e0 msgr2=0x7f6c94099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c940ce7e0 0x7f6c94099ec0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6c8802b8a0 tx=0x7f6c880046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 shutdown_connections 2023-12-12T09:53:05.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6c7406ae40 0x7f6c7406d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9415af50 0x7f6c9409a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c940ce7e0 0x7f6c94099ec0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 --2- 172.21.15.72:0/3154185918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9406a460 0x7f6c94153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 >> 172.21.15.72:0/3154185918 conn(0x7f6c940cee70 msgr2=0x7f6c9415beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:05.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 shutdown_connections 2023-12-12T09:53:05.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.707+0000 7f6c98f05640 1 -- 172.21.15.72:0/3154185918 wait complete. 2023-12-12T09:53:05.728 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:05.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.943+0000 7f402b1fb640 1 Processor -- start 2023-12-12T09:53:05.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.943+0000 7f402b1fb640 1 -- start start 2023-12-12T09:53:05.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f402b1fb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f402406dc10 0x7f4024168310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f402b1fb640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4024143bf0 con 0x7f4024143810 2023-12-12T09:53:05.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f402b1fb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40240ae520 con 0x7f40240cd7d0 2023-12-12T09:53:05.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f402b1fb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40240ae800 con 0x7f402406dc10 2023-12-12T09:53:05.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f4028f97640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4024143810 0x7f40240ada80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39940/0 (socket says 172.21.15.72:39940) 2023-12-12T09:53:05.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f4028f97640 1 -- 172.21.15.72:0/2963821238 learned_addr learned my addr 172.21.15.72:0/2963821238 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:05.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f4029798640 1 --2- 172.21.15.72:0/2963821238 >> v2:172.21.15.113:3300/0 conn(0x7f402406dc10 0x7f4024168310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691190637 0 0) 0x7f4024143bf0 con 0x7f4024143810 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.946+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4014003520 con 0x7f4024143810 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3757225752 0 0) 0x7f4014003520 con 0x7f4024143810 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 >> v2:172.21.15.113:3300/0 conn(0x7f402406dc10 msgr2=0x7f4024168310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 --2- 172.21.15.72:0/2963821238 >> v2:172.21.15.113:3300/0 conn(0x7f402406dc10 0x7f4024168310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 >> v1:172.21.15.113:6789/0 conn(0x7f40240cd7d0 legacy=0x7f402406d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240cccf0 con 0x7f4024143810 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f4018001670 con 0x7f4024143810 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 >> v1:172.21.15.72:6789/0 conn(0x7f4024143810 legacy=0x7f40240ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140039c0 0x7f4014003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40140047c0 0x7f4014004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4014005180 0x7f401400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40240ae800 con 0x7f40140047c0 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401400c050 con 0x7f40140039c0 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401400c210 con 0x7f4014005180 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.947+0000 7f4023fff640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40140047c0 0x7f4014004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4028f97640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140039c0 0x7f4014003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4029798640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4014005180 0x7f401400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4023fff640 1 -- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4014005180 msgr2=0x7f401400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4023fff640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4014005180 0x7f401400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4023fff640 1 -- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140039c0 msgr2=0x7f4014003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4023fff640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140039c0 0x7f4014003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4029798640 1 -- 172.21.15.72:0/2963821238 reap_dead start 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4023fff640 1 -- 172.21.15.72:0/2963821238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4014003520 con 0x7f40140047c0 2023-12-12T09:53:05.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.948+0000 7f4028f97640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140039c0 0x7f4014003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:05.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.949+0000 7f4023fff640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40140047c0 0x7f4014004c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4010003db0 tx=0x7f40140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e355f652b3ef4e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.949+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010032600 con 0x7f40140047c0 2023-12-12T09:53:05.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.949+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401002e050 con 0x7f40140047c0 2023-12-12T09:53:05.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.949+0000 7f40237fe640 1 -- 172.21.15.72:0/2963821238 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40100329e0 con 0x7f40140047c0 2023-12-12T09:53:05.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.950+0000 7f402b1fb640 1 -- 172.21.15.72:0/2963821238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40140047c0 msgr2=0x7f4014004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.950+0000 7f402b1fb640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40140047c0 0x7f4014004c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4010003db0 tx=0x7f40140057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.950+0000 7f402b1fb640 1 -- 172.21.15.72:0/2963821238 shutdown_connections 2023-12-12T09:53:05.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.950+0000 7f402b1fb640 1 --2- 172.21.15.72:0/2963821238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40140047c0 0x7f4014004c40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.950+0000 7f402b1fb640 1 -- 172.21.15.72:0/2963821238 >> 172.21.15.72:0/2963821238 conn(0x7f40240cb850 msgr2=0x7f40240cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:05.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.950+0000 7f402b1fb640 1 -- 172.21.15.72:0/2963821238 shutdown_connections 2023-12-12T09:53:05.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.950+0000 7f402b1fb640 1 -- 172.21.15.72:0/2963821238 wait complete. 2023-12-12T09:53:05.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.952+0000 7f402b1fb640 1 Processor -- start 2023-12-12T09:53:05.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.952+0000 7f402b1fb640 1 -- start start 2023-12-12T09:53:05.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f402b1fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140047c0 0x7f40241e7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f402b1fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40240b3c50 0x7f4024140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f402b1fb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40240b40d0 0x7f40241407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f402b1fb640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241e2c10 con 0x7f40240b40d0 2023-12-12T09:53:05.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f402b1fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241e60a0 con 0x7f40240b3c50 2023-12-12T09:53:05.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f402b1fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40241e6380 con 0x7f40140047c0 2023-12-12T09:53:05.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4029798640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40240b40d0 0x7f40241407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4029798640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40240b40d0 0x7f40241407d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36078/0 (socket says 172.21.15.72:36078) 2023-12-12T09:53:05.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4029798640 1 -- 172.21.15.72:0/2334831890 learned_addr learned my addr 172.21.15.72:0/2334831890 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:05.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4028f97640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140047c0 0x7f40241e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4029798640 1 -- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140047c0 msgr2=0x7f40241e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4029798640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140047c0 0x7f40241e7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4023fff640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40240b3c50 0x7f4024140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4029798640 1 -- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40240b3c50 msgr2=0x7f4024140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4029798640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40240b3c50 0x7f4024140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4029798640 1 -- 172.21.15.72:0/2334831890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40241e6660 con 0x7f40240b40d0 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.953+0000 7f4023fff640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40240b3c50 0x7f4024140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.954+0000 7f4029798640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40240b40d0 0x7f40241407d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f402406e4f0 tx=0x7f401c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.954+0000 7f40217fa640 1 -- 172.21.15.72:0/2334831890 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c016620 con 0x7f40240b40d0 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.954+0000 7f40217fa640 1 -- 172.21.15.72:0/2334831890 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c0167a0 con 0x7f40240b40d0 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.954+0000 7f40217fa640 1 -- 172.21.15.72:0/2334831890 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c016b50 con 0x7f40240b40d0 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.954+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40241e6940 con 0x7f40240b40d0 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.954+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40241e5470 con 0x7f40240b40d0 2023-12-12T09:53:05.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.956+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40240cc650 con 0x7f40240b40d0 2023-12-12T09:53:05.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.956+0000 7f40217fa640 1 -- 172.21.15.72:0/2334831890 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f401c016cb0 con 0x7f40240b40d0 2023-12-12T09:53:05.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.957+0000 7f40217fa640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f400806ae40 0x7f400806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.957+0000 7f40217fa640 1 -- 172.21.15.72:0/2334831890 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f401c08ab00 con 0x7f40240b40d0 2023-12-12T09:53:05.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.958+0000 7f4028f97640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f400806ae40 0x7f400806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.958+0000 7f4028f97640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f400806ae40 0x7f400806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4018000fd0 tx=0x7f4018009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:05.961+0000 7f40217fa640 1 -- 172.21.15.72:0/2334831890 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f401c08ce60 con 0x7f40240b40d0 2023-12-12T09:53:06.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.144+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f40241e6e60 con 0x7f40240b40d0 2023-12-12T09:53:06.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.145+0000 7f40217fa640 1 -- 172.21.15.72:0/2334831890 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f40241e6e60 con 0x7f40240b40d0 2023-12-12T09:53:06.147 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.147+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f400806ae40 msgr2=0x7f400806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.147+0000 7f402b1fb640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f400806ae40 0x7f400806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4018000fd0 tx=0x7f4018009040 comp rx=0 tx=0).stop 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.147+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40240b40d0 msgr2=0x7f40241407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.147+0000 7f402b1fb640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40240b40d0 0x7f40241407d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f402406e4f0 tx=0x7f401c00f520 comp rx=0 tx=0).stop 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.148+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 shutdown_connections 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.148+0000 7f402b1fb640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f400806ae40 0x7f400806d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.148+0000 7f402b1fb640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40240b40d0 0x7f40241407d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.148+0000 7f402b1fb640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40240b3c50 0x7f4024140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.148+0000 7f402b1fb640 1 --2- 172.21.15.72:0/2334831890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140047c0 0x7f40241e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.148+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 >> 172.21.15.72:0/2334831890 conn(0x7f40240cb850 msgr2=0x7f40240b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:06.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.148+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 shutdown_connections 2023-12-12T09:53:06.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.148+0000 7f402b1fb640 1 -- 172.21.15.72:0/2334831890 wait complete. 2023-12-12T09:53:06.167 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:06.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.374+0000 7f4101b68640 1 Processor -- start 2023-12-12T09:53:06.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.374+0000 7f4101b68640 1 -- start start 2023-12-12T09:53:06.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.375+0000 7f4101b68640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40fc1467c0 0x7f40fc0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.375+0000 7f4101b68640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f40fc146ba0 0x7f40fc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.375+0000 7f4101b68640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc06d510 con 0x7f40fc06c7e0 2023-12-12T09:53:06.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.375+0000 7f4101b68640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc0ac860 con 0x7f40fc146ba0 2023-12-12T09:53:06.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.375+0000 7f4101b68640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc06d7f0 con 0x7f40fc1467c0 2023-12-12T09:53:06.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40faffd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f40fc146ba0 0x7f40fc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40faffd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f40fc146ba0 0x7f40fc0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36094/0 (socket says 172.21.15.72:36094) 2023-12-12T09:53:06.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40faffd640 1 -- 172.21.15.72:0/1153823001 learned_addr learned my addr 172.21.15.72:0/1153823001 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:06.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40fb7fe640 1 --2- 172.21.15.72:0/1153823001 >> v2:172.21.15.113:3300/0 conn(0x7f40fc1467c0 0x7f40fc0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40faffd640 1 -- 172.21.15.72:0/1153823001 >> v2:172.21.15.113:3300/0 conn(0x7f40fc1467c0 msgr2=0x7f40fc0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40faffd640 1 --2- 172.21.15.72:0/1153823001 >> v2:172.21.15.113:3300/0 conn(0x7f40fc1467c0 0x7f40fc0cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40faffd640 1 -- 172.21.15.72:0/1153823001 >> v1:172.21.15.113:6789/0 conn(0x7f40fc06c7e0 legacy=0x7f40fc06cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40faffd640 1 -- 172.21.15.72:0/1153823001 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc06dad0 con 0x7f40fc146ba0 2023-12-12T09:53:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.376+0000 7f40fb7fe640 1 --2- 172.21.15.72:0/1153823001 >> v2:172.21.15.113:3300/0 conn(0x7f40fc1467c0 0x7f40fc0cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.377+0000 7f40faffd640 1 --2- 172.21.15.72:0/1153823001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc146ba0 0x7f40fc0cce30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0cc830 tx=0x7f40e402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3318f4288a62be7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f40fa7fc640 1 -- 172.21.15.72:0/1153823001 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4031890 con 0x7f40fc146ba0 2023-12-12T09:53:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f40fa7fc640 1 -- 172.21.15.72:0/1153823001 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e40319f0 con 0x7f40fc146ba0 2023-12-12T09:53:06.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f40fa7fc640 1 -- 172.21.15.72:0/1153823001 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4031d80 con 0x7f40fc146ba0 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f4101b68640 1 -- 172.21.15.72:0/1153823001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc146ba0 msgr2=0x7f40fc0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f4101b68640 1 --2- 172.21.15.72:0/1153823001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc146ba0 0x7f40fc0cce30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0cc830 tx=0x7f40e402b930 comp rx=0 tx=0).stop 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f4101b68640 1 -- 172.21.15.72:0/1153823001 shutdown_connections 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f4101b68640 1 --2- 172.21.15.72:0/1153823001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc146ba0 0x7f40fc0cce30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f4101b68640 1 --2- 172.21.15.72:0/1153823001 >> v2:172.21.15.113:3300/0 conn(0x7f40fc1467c0 0x7f40fc0cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f4101b68640 1 -- 172.21.15.72:0/1153823001 >> 172.21.15.72:0/1153823001 conn(0x7f40fc0b0a90 msgr2=0x7f40fc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f4101b68640 1 -- 172.21.15.72:0/1153823001 shutdown_connections 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.378+0000 7f4101b68640 1 -- 172.21.15.72:0/1153823001 wait complete. 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.380+0000 7f4101b68640 1 Processor -- start 2023-12-12T09:53:06.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.380+0000 7f4101b68640 1 -- start start 2023-12-12T09:53:06.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f4101b68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40fc06c7e0 0x7f40fc142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f4101b68640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc1467c0 0x7f40fc13f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f4101b68640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40fc146ba0 0x7f40fc144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f4101b68640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc1e7e90 con 0x7f40fc1467c0 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f4101b68640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc143690 con 0x7f40fc146ba0 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f4101b68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0b43a0 con 0x7f40fc06c7e0 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f40faffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc1467c0 0x7f40fc13f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f40faffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc1467c0 0x7f40fc13f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36106/0 (socket says 172.21.15.72:36106) 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.381+0000 7f40faffd640 1 -- 172.21.15.72:0/3968916219 learned_addr learned my addr 172.21.15.72:0/3968916219 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40fb7fe640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40fc06c7e0 0x7f40fc142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40fbfff640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40fc146ba0 0x7f40fc144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40faffd640 1 -- 172.21.15.72:0/3968916219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40fc06c7e0 msgr2=0x7f40fc142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40faffd640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40fc06c7e0 0x7f40fc142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40faffd640 1 -- 172.21.15.72:0/3968916219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40fc146ba0 msgr2=0x7f40fc144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40faffd640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40fc146ba0 0x7f40fc144920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40faffd640 1 -- 172.21.15.72:0/3968916219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e40094d0 con 0x7f40fc1467c0 2023-12-12T09:53:06.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40faffd640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc1467c0 0x7f40fc13f370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40f3fff640 1 -- 172.21.15.72:0/3968916219 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4031840 con 0x7f40fc1467c0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40f3fff640 1 -- 172.21.15.72:0/3968916219 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e40443f0 con 0x7f40fc1467c0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f40f3fff640 1 -- 172.21.15.72:0/3968916219 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e40447a0 con 0x7f40fc1467c0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.382+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc0b4680 con 0x7f40fc1467c0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.383+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc0b4c40 con 0x7f40fc1467c0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.384+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc0b4e30 con 0x7f40fc1467c0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.385+0000 7f40f3fff640 1 -- 172.21.15.72:0/3968916219 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40e404d450 con 0x7f40fc1467c0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.385+0000 7f40f3fff640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f40dc06ae40 0x7f40dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.386+0000 7f40f3fff640 1 -- 172.21.15.72:0/3968916219 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f40e40ac210 con 0x7f40fc1467c0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.386+0000 7f40fb7fe640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f40dc06ae40 0x7f40dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.386+0000 7f40fb7fe640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f40dc06ae40 0x7f40dc06d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f40ec008850 tx=0x7f40ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:06.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.390+0000 7f40f3fff640 1 -- 172.21.15.72:0/3968916219 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40e4076ef0 con 0x7f40fc1467c0 2023-12-12T09:53:06.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.584+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40fc0745d0 con 0x7f40fc1467c0 2023-12-12T09:53:06.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.586+0000 7f40f3fff640 1 -- 172.21.15.72:0/3968916219 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7f40e407a510 con 0x7f40fc1467c0 2023-12-12T09:53:06.589 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:06.590 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:06.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f40dc06ae40 msgr2=0x7f40dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f40dc06ae40 0x7f40dc06d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f40ec008850 tx=0x7f40ec008040 comp rx=0 tx=0).stop 2023-12-12T09:53:06.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc1467c0 msgr2=0x7f40fc13f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc1467c0 0x7f40fc13f370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e40046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 shutdown_connections 2023-12-12T09:53:06.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f40dc06ae40 0x7f40dc06d300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40fc146ba0 0x7f40fc144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40fc1467c0 0x7f40fc13f370 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 --2- 172.21.15.72:0/3968916219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40fc06c7e0 0x7f40fc142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.589+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 >> 172.21.15.72:0/3968916219 conn(0x7f40fc0b0a90 msgr2=0x7f40fc1576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:06.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.590+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 shutdown_connections 2023-12-12T09:53:06.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.590+0000 7f4101b68640 1 -- 172.21.15.72:0/3968916219 wait complete. 2023-12-12T09:53:06.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:06.612 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:06.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.823+0000 7efc8210c640 1 Processor -- start 2023-12-12T09:53:06.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.823+0000 7efc8210c640 1 -- start start 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.824+0000 7efc8210c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efc7c15d0e0 0x7efc7c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc8210c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c0c92e0 0x7efc7c0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc8210c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc7c160dc0 con 0x7efc7c072210 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc8210c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7c161c40 con 0x7efc7c15d0e0 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc8210c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c070280 con 0x7efc7c0c92e0 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7b7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efc7c15d0e0 0x7efc7c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7b7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efc7c15d0e0 0x7efc7c0ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36110/0 (socket says 172.21.15.72:36110) 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7affd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7efc7c072210 0x7efc7c161530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39946/0 (socket says 172.21.15.72:39946) 2023-12-12T09:53:06.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7b7fe640 1 -- 172.21.15.72:0/2928721043 learned_addr learned my addr 172.21.15.72:0/2928721043 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:06.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7bfff640 1 --2- 172.21.15.72:0/2928721043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c0c92e0 0x7efc7c0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7b7fe640 1 -- 172.21.15.72:0/2928721043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c0c92e0 msgr2=0x7efc7c0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:06.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7b7fe640 1 --2- 172.21.15.72:0/2928721043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c0c92e0 0x7efc7c0c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7b7fe640 1 -- 172.21.15.72:0/2928721043 >> v1:172.21.15.72:6789/0 conn(0x7efc7c072210 legacy=0x7efc7c161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.825+0000 7efc7b7fe640 1 -- 172.21.15.72:0/2928721043 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c070560 con 0x7efc7c15d0e0 2023-12-12T09:53:06.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.826+0000 7efc7bfff640 1 --2- 172.21.15.72:0/2928721043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c0c92e0 0x7efc7c0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:06.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.827+0000 7efc7b7fe640 1 --2- 172.21.15.72:0/2928721043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c15d0e0 0x7efc7c0ce3a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009870 tx=0x7efc6c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b14f51e8d7dd3bf server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:06.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.827+0000 7efc7a7fc640 1 -- 172.21.15.72:0/2928721043 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c030840 con 0x7efc7c15d0e0 2023-12-12T09:53:06.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.827+0000 7efc7a7fc640 1 -- 172.21.15.72:0/2928721043 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c0309a0 con 0x7efc7c15d0e0 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.827+0000 7efc7a7fc640 1 -- 172.21.15.72:0/2928721043 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c030d30 con 0x7efc7c15d0e0 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.827+0000 7efc8210c640 1 -- 172.21.15.72:0/2928721043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c15d0e0 msgr2=0x7efc7c0ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.827+0000 7efc8210c640 1 --2- 172.21.15.72:0/2928721043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c15d0e0 0x7efc7c0ce3a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009870 tx=0x7efc6c02b980 comp rx=0 tx=0).stop 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.828+0000 7efc8210c640 1 -- 172.21.15.72:0/2928721043 shutdown_connections 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.828+0000 7efc8210c640 1 --2- 172.21.15.72:0/2928721043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c0c92e0 0x7efc7c0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.828+0000 7efc8210c640 1 --2- 172.21.15.72:0/2928721043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c15d0e0 0x7efc7c0ce3a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.828+0000 7efc8210c640 1 -- 172.21.15.72:0/2928721043 >> 172.21.15.72:0/2928721043 conn(0x7efc7c0cee70 msgr2=0x7efc7c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.828+0000 7efc8210c640 1 -- 172.21.15.72:0/2928721043 shutdown_connections 2023-12-12T09:53:06.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.828+0000 7efc8210c640 1 -- 172.21.15.72:0/2928721043 wait complete. 2023-12-12T09:53:06.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.829+0000 7efc8210c640 1 Processor -- start 2023-12-12T09:53:06.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.830+0000 7efc8210c640 1 -- start start 2023-12-12T09:53:06.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.830+0000 7efc8210c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7c072210 0x7efc7c15aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.830+0000 7efc8210c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c0c92e0 0x7efc7c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc8210c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c15d0e0 0x7efc7c0adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc8210c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c158930 con 0x7efc7c0c92e0 2023-12-12T09:53:06.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc8210c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c0ce220 con 0x7efc7c072210 2023-12-12T09:53:06.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc8210c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c0ad3e0 con 0x7efc7c15d0e0 2023-12-12T09:53:06.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c0c92e0 0x7efc7c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c0c92e0 0x7efc7c0cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36116/0 (socket says 172.21.15.72:36116) 2023-12-12T09:53:06.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7affd640 1 -- 172.21.15.72:0/532844170 learned_addr learned my addr 172.21.15.72:0/532844170 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:06.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7affd640 1 -- 172.21.15.72:0/532844170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c15d0e0 msgr2=0x7efc7c0adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7affd640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c15d0e0 0x7efc7c0adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7affd640 1 -- 172.21.15.72:0/532844170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7c072210 msgr2=0x7efc7c15aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:06.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7b7fe640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7c072210 0x7efc7c15aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7affd640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7c072210 0x7efc7c15aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:06.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.831+0000 7efc7affd640 1 -- 172.21.15.72:0/532844170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c0094d0 con 0x7efc7c0c92e0 2023-12-12T09:53:06.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.832+0000 7efc7affd640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c0c92e0 0x7efc7c0cdce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc7000c450 tx=0x7efc7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:06.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.832+0000 7efc63fff640 1 -- 172.21.15.72:0/532844170 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70007480 con 0x7efc7c0c92e0 2023-12-12T09:53:06.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.832+0000 7efc63fff640 1 -- 172.21.15.72:0/532844170 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc700075e0 con 0x7efc7c0c92e0 2023-12-12T09:53:06.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.832+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7c0ce450 con 0x7efc7c0c92e0 2023-12-12T09:53:06.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.832+0000 7efc63fff640 1 -- 172.21.15.72:0/532844170 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70004a00 con 0x7efc7c0c92e0 2023-12-12T09:53:06.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.832+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c0ad680 con 0x7efc7c0c92e0 2023-12-12T09:53:06.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.833+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7c0ad240 con 0x7efc7c0c92e0 2023-12-12T09:53:06.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.834+0000 7efc63fff640 1 -- 172.21.15.72:0/532844170 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc70004b60 con 0x7efc7c0c92e0 2023-12-12T09:53:06.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.835+0000 7efc63fff640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc6806ae40 0x7efc6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:06.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.835+0000 7efc63fff640 1 -- 172.21.15.72:0/532844170 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efc70088dc0 con 0x7efc7c0c92e0 2023-12-12T09:53:06.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.835+0000 7efc7b7fe640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc6806ae40 0x7efc6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:06.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.836+0000 7efc7b7fe640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc6806ae40 0x7efc6806d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efc6c036000 tx=0x7efc6c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:06.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:06.839+0000 7efc63fff640 1 -- 172.21.15.72:0/532844170 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc7c0ad240 con 0x7efc7c0c92e0 2023-12-12T09:53:07.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.027+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc7c0ad240 con 0x7efc7c0c92e0 2023-12-12T09:53:07.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.028+0000 7efc63fff640 1 -- 172.21.15.72:0/532844170 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc7c0ad240 con 0x7efc7c0c92e0 2023-12-12T09:53:07.031 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:07.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.030+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc6806ae40 msgr2=0x7efc6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.030+0000 7efc8210c640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc6806ae40 0x7efc6806d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efc6c036000 tx=0x7efc6c03b040 comp rx=0 tx=0).stop 2023-12-12T09:53:07.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c0c92e0 msgr2=0x7efc7c0cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c0c92e0 0x7efc7c0cdce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc7000c450 tx=0x7efc7000c910 comp rx=0 tx=0).stop 2023-12-12T09:53:07.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 shutdown_connections 2023-12-12T09:53:07.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc6806ae40 0x7efc6806d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc7c15d0e0 0x7efc7c0adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc7c0c92e0 0x7efc7c0cdce0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 --2- 172.21.15.72:0/532844170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7c072210 0x7efc7c15aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 >> 172.21.15.72:0/532844170 conn(0x7efc7c0cee70 msgr2=0x7efc7c15d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:07.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 shutdown_connections 2023-12-12T09:53:07.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.031+0000 7efc8210c640 1 -- 172.21.15.72:0/532844170 wait complete. 2023-12-12T09:53:07.052 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:07.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.264+0000 7f805e3b0640 1 Processor -- start 2023-12-12T09:53:07.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.265+0000 7f805e3b0640 1 -- start start 2023-12-12T09:53:07.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.266+0000 7f805e3b0640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f80581609b0 0x7f80580d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.266+0000 7f805e3b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.266+0000 7f805e3b0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805815dd60 con 0x7f80580d0060 2023-12-12T09:53:07.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.266+0000 7f805e3b0640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80580d04f0 con 0x7f80581609b0 2023-12-12T09:53:07.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.266+0000 7f805e3b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8058160d90 con 0x7f80580d1220 2023-12-12T09:53:07.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.266+0000 7f8057fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f80581609b0 0x7f80580d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.266+0000 7f8057fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f80581609b0 0x7f80580d0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36120/0 (socket says 172.21.15.72:36120) 2023-12-12T09:53:07.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.266+0000 7f8057fff640 1 -- 172.21.15.72:0/3410851484 learned_addr learned my addr 172.21.15.72:0/3410851484 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:07.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.267+0000 7f8057fff640 1 -- 172.21.15.72:0/3410851484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 msgr2=0x7f80580cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:07.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.267+0000 7f80577fe640 1 --2- 172.21.15.72:0/3410851484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.267+0000 7f8057fff640 1 --2- 172.21.15.72:0/3410851484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.267+0000 7f8057fff640 1 -- 172.21.15.72:0/3410851484 >> v1:172.21.15.113:6789/0 conn(0x7f80580d0060 legacy=0x7f805815cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:07.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.267+0000 7f8057fff640 1 -- 172.21.15.72:0/3410851484 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805815e040 con 0x7f80581609b0 2023-12-12T09:53:07.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.267+0000 7f80577fe640 1 --2- 172.21.15.72:0/3410851484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.268+0000 7f8057fff640 1 --2- 172.21.15.72:0/3410851484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80581609b0 0x7f80580d0ce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8048009e80 tx=0x7f804802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cef2724fe2b0817 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.268+0000 7f8056ffd640 1 -- 172.21.15.72:0/3410851484 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804802ca50 con 0x7f80581609b0 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.268+0000 7f8056ffd640 1 -- 172.21.15.72:0/3410851484 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804802cbb0 con 0x7f80581609b0 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.268+0000 7f8056ffd640 1 -- 172.21.15.72:0/3410851484 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80480343f0 con 0x7f80581609b0 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.269+0000 7f805e3b0640 1 -- 172.21.15.72:0/3410851484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80581609b0 msgr2=0x7f80580d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.269+0000 7f805e3b0640 1 --2- 172.21.15.72:0/3410851484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80581609b0 0x7f80580d0ce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8048009e80 tx=0x7f804802beb0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.269+0000 7f805e3b0640 1 -- 172.21.15.72:0/3410851484 shutdown_connections 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.269+0000 7f805e3b0640 1 --2- 172.21.15.72:0/3410851484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.269+0000 7f805e3b0640 1 --2- 172.21.15.72:0/3410851484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80581609b0 0x7f80580d0ce0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.269+0000 7f805e3b0640 1 -- 172.21.15.72:0/3410851484 >> 172.21.15.72:0/3410851484 conn(0x7f805809c450 msgr2=0x7f80580d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.269+0000 7f805e3b0640 1 -- 172.21.15.72:0/3410851484 shutdown_connections 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.269+0000 7f805e3b0640 1 -- 172.21.15.72:0/3410851484 wait complete. 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.271+0000 7f805e3b0640 1 Processor -- start 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.271+0000 7f805e3b0640 1 -- start start 2023-12-12T09:53:07.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f805e3b0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80580d0060 0x7f805815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f805e3b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f805e3b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80581609b0 0x7f80580c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f805e3b0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b8380 con 0x7f80580d0060 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f805e3b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b8660 con 0x7f80581609b0 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f805e3b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b8940 con 0x7f80580d1220 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f8057fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80580d0060 0x7f805815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f8057fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80580d0060 0x7f805815b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36130/0 (socket says 172.21.15.72:36130) 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.272+0000 7f8057fff640 1 -- 172.21.15.72:0/2085580291 learned_addr learned my addr 172.21.15.72:0/2085580291 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f80577fe640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8057fff640 1 -- 172.21.15.72:0/2085580291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 msgr2=0x7f80580b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8057fff640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8057fff640 1 -- 172.21.15.72:0/2085580291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80581609b0 msgr2=0x7f80580c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8057fff640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80581609b0 0x7f80580c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8057fff640 1 -- 172.21.15.72:0/2085580291 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048009b30 con 0x7f80580d0060 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8057fff640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80580d0060 0x7f805815b7c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8048036040 tx=0x7f8048004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8054ff9640 1 -- 172.21.15.72:0/2085580291 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804803e450 con 0x7f80580d0060 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8054ff9640 1 -- 172.21.15.72:0/2085580291 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804803e5b0 con 0x7f80580d0060 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f8054ff9640 1 -- 172.21.15.72:0/2085580291 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804803e960 con 0x7f80580d0060 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80580cb0d0 con 0x7f80580d0060 2023-12-12T09:53:07.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.273+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80581e8a90 con 0x7f80580d0060 2023-12-12T09:53:07.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.274+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80580745d0 con 0x7f80580d0060 2023-12-12T09:53:07.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.275+0000 7f8054ff9640 1 -- 172.21.15.72:0/2085580291 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f804803eac0 con 0x7f80580d0060 2023-12-12T09:53:07.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.276+0000 7f8054ff9640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f803806ae40 0x7f803806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.276+0000 7f8054ff9640 1 -- 172.21.15.72:0/2085580291 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f80480ac800 con 0x7f80580d0060 2023-12-12T09:53:07.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.277+0000 7f80577fe640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f803806ae40 0x7f803806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.277+0000 7f80577fe640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f803806ae40 0x7f803806d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f80580caa80 tx=0x7f8040008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:07.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.280+0000 7f8054ff9640 1 -- 172.21.15.72:0/2085580291 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8048077580 con 0x7f80580d0060 2023-12-12T09:53:07.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.470+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80580cb310 con 0x7f80580d0060 2023-12-12T09:53:07.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.471+0000 7f8054ff9640 1 -- 172.21.15.72:0/2085580291 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80580cb310 con 0x7f80580d0060 2023-12-12T09:53:07.474 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:07.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f803806ae40 msgr2=0x7f803806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f803806ae40 0x7f803806d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f80580caa80 tx=0x7f8040008040 comp rx=0 tx=0).stop 2023-12-12T09:53:07.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80580d0060 msgr2=0x7f805815b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80580d0060 0x7f805815b7c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8048036040 tx=0x7f8048004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:07.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 shutdown_connections 2023-12-12T09:53:07.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f803806ae40 0x7f803806d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80581609b0 0x7f80580c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80580d1220 0x7f80580b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 --2- 172.21.15.72:0/2085580291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80580d0060 0x7f805815b7c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 >> 172.21.15.72:0/2085580291 conn(0x7f805809c450 msgr2=0x7f805815f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:07.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 shutdown_connections 2023-12-12T09:53:07.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.474+0000 7f805e3b0640 1 -- 172.21.15.72:0/2085580291 wait complete. 2023-12-12T09:53:07.495 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:07.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.714+0000 7f6bd3368640 1 Processor -- start 2023-12-12T09:53:07.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.714+0000 7f6bd3368640 1 -- start start 2023-12-12T09:53:07.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd3368640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc15af50 0x7f6bcc06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd3368640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6bcc06a460 0x7f6bcc06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd3368640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc071cd0 con 0x7f6bcc0ce7e0 2023-12-12T09:53:07.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd3368640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc06ae30 con 0x7f6bcc06a460 2023-12-12T09:53:07.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd3368640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc0717e0 con 0x7f6bcc15af50 2023-12-12T09:53:07.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd0903640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6bcc0ce7e0 0x7f6bcc15ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39962/0 (socket says 172.21.15.72:39962) 2023-12-12T09:53:07.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd0903640 1 -- 172.21.15.72:0/3882307894 learned_addr learned my addr 172.21.15.72:0/3882307894 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:07.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd1104640 1 --2- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc15af50 0x7f6bcc06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.716+0000 7f6bd1905640 1 --2- 172.21.15.72:0/3882307894 >> v2:172.21.15.113:3300/0 conn(0x7f6bcc06a460 0x7f6bcc06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.717+0000 7f6bc3fff640 1 -- 172.21.15.72:0/3882307894 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658703407 0 0) 0x7f6bcc071cd0 con 0x7f6bcc0ce7e0 2023-12-12T09:53:07.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.717+0000 7f6bc3fff640 1 -- 172.21.15.72:0/3882307894 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb8003520 con 0x7f6bcc0ce7e0 2023-12-12T09:53:07.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.717+0000 7f6bd1905640 1 -- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc15af50 msgr2=0x7f6bcc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:07.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.717+0000 7f6bd1905640 1 --2- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc15af50 0x7f6bcc06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.717+0000 7f6bc3fff640 1 -- 172.21.15.72:0/3882307894 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2283135786 0 0) 0x7f6bb8003520 con 0x7f6bcc0ce7e0 2023-12-12T09:53:07.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.717+0000 7f6bd1905640 1 -- 172.21.15.72:0/3882307894 >> v1:172.21.15.72:6789/0 conn(0x7f6bcc0ce7e0 legacy=0x7f6bcc15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.717+0000 7f6bd1905640 1 -- 172.21.15.72:0/3882307894 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc06c6b0 con 0x7f6bcc06a460 2023-12-12T09:53:07.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.717+0000 7f6bd1104640 1 --2- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc15af50 0x7f6bcc06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:53:07.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.719+0000 7f6bd1905640 1 --2- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc06a460 0x7f6bcc06a8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40097a0 tx=0x7f6bc402b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3165c2b35a3abf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:07.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.719+0000 7f6bc3fff640 1 -- 172.21.15.72:0/3882307894 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc001670 con 0x7f6bcc06a460 2023-12-12T09:53:07.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.719+0000 7f6bc3fff640 1 -- 172.21.15.72:0/3882307894 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc402e050 con 0x7f6bcc06a460 2023-12-12T09:53:07.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.719+0000 7f6bc3fff640 1 -- 172.21.15.72:0/3882307894 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4030730 con 0x7f6bcc06a460 2023-12-12T09:53:07.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.720+0000 7f6bd3368640 1 -- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc06a460 msgr2=0x7f6bcc06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.720+0000 7f6bd3368640 1 --2- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc06a460 0x7f6bcc06a8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40097a0 tx=0x7f6bc402b660 comp rx=0 tx=0).stop 2023-12-12T09:53:07.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.720+0000 7f6bd3368640 1 -- 172.21.15.72:0/3882307894 shutdown_connections 2023-12-12T09:53:07.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.720+0000 7f6bd3368640 1 --2- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc06a460 0x7f6bcc06a8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.720+0000 7f6bd3368640 1 --2- 172.21.15.72:0/3882307894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc15af50 0x7f6bcc06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.720+0000 7f6bd3368640 1 -- 172.21.15.72:0/3882307894 >> 172.21.15.72:0/3882307894 conn(0x7f6bcc0cee70 msgr2=0x7f6bcc0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:07.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.720+0000 7f6bd3368640 1 -- 172.21.15.72:0/3882307894 shutdown_connections 2023-12-12T09:53:07.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.720+0000 7f6bd3368640 1 -- 172.21.15.72:0/3882307894 wait complete. 2023-12-12T09:53:07.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.722+0000 7f6bd3368640 1 Processor -- start 2023-12-12T09:53:07.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.723+0000 7f6bd3368640 1 -- start start 2023-12-12T09:53:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd3368640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc06a460 0x7f6bcc0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd3368640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc0ce7e0 0x7f6bcc14aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd3368640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bcc15af50 0x7f6bcc14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd3368640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc14b6e0 con 0x7f6bcc15af50 2023-12-12T09:53:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd3368640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc0cb270 con 0x7f6bcc0ce7e0 2023-12-12T09:53:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd3368640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc099c90 con 0x7f6bcc06a460 2023-12-12T09:53:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd1905640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bcc15af50 0x7f6bcc14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd1905640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bcc15af50 0x7f6bcc14b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36146/0 (socket says 172.21.15.72:36146) 2023-12-12T09:53:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.724+0000 7f6bd1905640 1 -- 172.21.15.72:0/1802543964 learned_addr learned my addr 172.21.15.72:0/1802543964 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd1104640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc06a460 0x7f6bcc0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd1905640 1 -- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc06a460 msgr2=0x7f6bcc0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd1905640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc06a460 0x7f6bcc0c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd1905640 1 -- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc0ce7e0 msgr2=0x7f6bcc14aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd0903640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc0ce7e0 0x7f6bcc14aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd1905640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc0ce7e0 0x7f6bcc14aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd1905640 1 -- 172.21.15.72:0/1802543964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc8009a70 con 0x7f6bcc15af50 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd1104640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc06a460 0x7f6bcc0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd0903640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc0ce7e0 0x7f6bcc14aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd1905640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bcc15af50 0x7f6bcc14b1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40097a0 tx=0x7f6bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bc1ffb640 1 -- 172.21.15.72:0/1802543964 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc403b810 con 0x7f6bcc15af50 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bc1ffb640 1 -- 172.21.15.72:0/1802543964 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc403b970 con 0x7f6bcc15af50 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bc1ffb640 1 -- 172.21.15.72:0/1802543964 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc403bce0 con 0x7f6bcc15af50 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.725+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc4009400 con 0x7f6bcc15af50 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.726+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc09c810 con 0x7f6bcc15af50 2023-12-12T09:53:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.727+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc074450 con 0x7f6bcc15af50 2023-12-12T09:53:07.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.728+0000 7f6bc1ffb640 1 -- 172.21.15.72:0/1802543964 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bc4039040 con 0x7f6bcc15af50 2023-12-12T09:53:07.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.728+0000 7f6bc1ffb640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6bb006ae40 0x7f6bb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:07.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.728+0000 7f6bc1ffb640 1 -- 172.21.15.72:0/1802543964 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6bc40aaf80 con 0x7f6bcc15af50 2023-12-12T09:53:07.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.729+0000 7f6bd1104640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6bb006ae40 0x7f6bb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:07.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.729+0000 7f6bd1104640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6bb006ae40 0x7f6bb006d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009fd0 tx=0x7f6bc800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:07.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.734+0000 7f6bc1ffb640 1 -- 172.21.15.72:0/1802543964 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bc4075be0 con 0x7f6bcc15af50 2023-12-12T09:53:07.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.924+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6bcc09a280 con 0x7f6bcc15af50 2023-12-12T09:53:07.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.924+0000 7f6bc1ffb640 1 -- 172.21.15.72:0/1802543964 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6bc4006e10 con 0x7f6bcc15af50 2023-12-12T09:53:07.927 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:07.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6bb006ae40 msgr2=0x7f6bb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6bb006ae40 0x7f6bb006d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009fd0 tx=0x7f6bc800b040 comp rx=0 tx=0).stop 2023-12-12T09:53:07.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bcc15af50 msgr2=0x7f6bcc14b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:07.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bcc15af50 0x7f6bcc14b1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40097a0 tx=0x7f6bc40046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 shutdown_connections 2023-12-12T09:53:07.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6bb006ae40 0x7f6bb006d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bcc15af50 0x7f6bcc14b1a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bcc0ce7e0 0x7f6bcc14aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 --2- 172.21.15.72:0/1802543964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bcc06a460 0x7f6bcc0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:07.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 >> 172.21.15.72:0/1802543964 conn(0x7f6bcc0cee70 msgr2=0x7f6bcc15c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:07.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 shutdown_connections 2023-12-12T09:53:07.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:07.927+0000 7f6bd3368640 1 -- 172.21.15.72:0/1802543964 wait complete. 2023-12-12T09:53:07.947 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:08.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.168+0000 7fcca018a640 1 Processor -- start 2023-12-12T09:53:08.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.168+0000 7fcca018a640 1 -- start start 2023-12-12T09:53:08.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.170+0000 7fcca018a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcc9806dc10 0x7fcc98164f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.170+0000 7fcca018a640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc980b4980 con 0x7fcc98142de0 2023-12-12T09:53:08.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.170+0000 7fcca018a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc98071260 con 0x7fcc981432b0 2023-12-12T09:53:08.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.170+0000 7fcca018a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc980b4050 con 0x7fcc9806dc10 2023-12-12T09:53:08.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.170+0000 7fcc9df26640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fcc98142de0 0x7fcc980cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39964/0 (socket says 172.21.15.72:39964) 2023-12-12T09:53:08.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.170+0000 7fcc9df26640 1 -- 172.21.15.72:0/2704356523 learned_addr learned my addr 172.21.15.72:0/2704356523 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9e727640 1 --2- 172.21.15.72:0/2704356523 >> v2:172.21.15.113:3300/0 conn(0x7fcc9806dc10 0x7fcc98164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558077853 0 0) 0x7fcc980b4980 con 0x7fcc98142de0 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc84003520 con 0x7fcc98142de0 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 291194680 0 0) 0x7fcc84003520 con 0x7fcc98142de0 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 >> v2:172.21.15.113:3300/0 conn(0x7fcc9806dc10 msgr2=0x7fcc98164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 --2- 172.21.15.72:0/2704356523 >> v2:172.21.15.113:3300/0 conn(0x7fcc9806dc10 0x7fcc98164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 >> v1:172.21.15.113:6789/0 conn(0x7fcc981432b0 legacy=0x7fcc9806d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc980b4330 con 0x7fcc98142de0 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fcc94001670 con 0x7fcc98142de0 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 >> v1:172.21.15.72:6789/0 conn(0x7fcc98142de0 legacy=0x7fcc980cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84003970 0x7fcc84003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc840047c0 0x7fcc84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc84005180 0x7fcc8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980b4050 con 0x7fcc84005180 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8400c0b0 con 0x7fcc84003970 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8400c270 con 0x7fcc840047c0 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.171+0000 7fcc9e727640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc84005180 0x7fcc8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9df26640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84003970 0x7fcc84003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9d725640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc840047c0 0x7fcc84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9e727640 1 -- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc840047c0 msgr2=0x7fcc84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9e727640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc840047c0 0x7fcc84004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9e727640 1 -- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84003970 msgr2=0x7fcc84003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9e727640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84003970 0x7fcc84003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9e727640 1 -- 172.21.15.72:0/2704356523 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84003520 con 0x7fcc84005180 2023-12-12T09:53:08.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9df26640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84003970 0x7fcc84003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.172+0000 7fcc9d725640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc840047c0 0x7fcc84004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.173+0000 7fcc9e727640 1 -- 172.21.15.72:0/2704356523 reap_dead start 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.173+0000 7fcc9e727640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc84005180 0x7fcc8400b980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009fc0 tx=0x7fcc840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f22fe02a819b9ee server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.173+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9002cc30 con 0x7fcc84005180 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.173+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9000f040 con 0x7fcc84005180 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.173+0000 7fcc9cf24640 1 -- 172.21.15.72:0/2704356523 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90003990 con 0x7fcc84005180 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.174+0000 7fcca018a640 1 -- 172.21.15.72:0/2704356523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc84005180 msgr2=0x7fcc8400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.174+0000 7fcca018a640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc84005180 0x7fcc8400b980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009fc0 tx=0x7fcc840057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.174+0000 7fcca018a640 1 -- 172.21.15.72:0/2704356523 shutdown_connections 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.174+0000 7fcca018a640 1 --2- 172.21.15.72:0/2704356523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc84005180 0x7fcc8400b980 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.174+0000 7fcca018a640 1 -- 172.21.15.72:0/2704356523 >> 172.21.15.72:0/2704356523 conn(0x7fcc980cb850 msgr2=0x7fcc980cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.174+0000 7fcca018a640 1 -- 172.21.15.72:0/2704356523 shutdown_connections 2023-12-12T09:53:08.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.174+0000 7fcca018a640 1 -- 172.21.15.72:0/2704356523 wait complete. 2023-12-12T09:53:08.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.176+0000 7fcca018a640 1 Processor -- start 2023-12-12T09:53:08.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.176+0000 7fcca018a640 1 -- start start 2023-12-12T09:53:08.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcca018a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84005180 0x7fcc9813d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcca018a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc9813ce80 0x7fcc980a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcca018a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc980a8990 0x7fcc980a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcca018a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9813dd10 con 0x7fcc9813ce80 2023-12-12T09:53:08.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcca018a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9815d390 con 0x7fcc84005180 2023-12-12T09:53:08.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcca018a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9815a8c0 con 0x7fcc980a8990 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcc9d725640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc9813ce80 0x7fcc980a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcc9d725640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc9813ce80 0x7fcc980a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36178/0 (socket says 172.21.15.72:36178) 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.177+0000 7fcc9d725640 1 -- 172.21.15.72:0/2964481235 learned_addr learned my addr 172.21.15.72:0/2964481235 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc9df26640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84005180 0x7fcc9813d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc9e727640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc980a8990 0x7fcc980a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc9d725640 1 -- 172.21.15.72:0/2964481235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc980a8990 msgr2=0x7fcc980a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc9d725640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc980a8990 0x7fcc980a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc9d725640 1 -- 172.21.15.72:0/2964481235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84005180 msgr2=0x7fcc9813d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc9d725640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84005180 0x7fcc9813d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc9d725640 1 -- 172.21.15.72:0/2964481235 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9815cdd0 con 0x7fcc9813ce80 2023-12-12T09:53:08.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc9d725640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc9813ce80 0x7fcc980a8450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800d5c0 tx=0x7fcc8800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:08.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc8e7fc640 1 -- 172.21.15.72:0/2964481235 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88007220 con 0x7fcc9813ce80 2023-12-12T09:53:08.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc8e7fc640 1 -- 172.21.15.72:0/2964481235 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc880073a0 con 0x7fcc9813ce80 2023-12-12T09:53:08.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcc8e7fc640 1 -- 172.21.15.72:0/2964481235 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc880067d0 con 0x7fcc9813ce80 2023-12-12T09:53:08.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.178+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9815d0b0 con 0x7fcc9813ce80 2023-12-12T09:53:08.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.179+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc980ab2b0 con 0x7fcc9813ce80 2023-12-12T09:53:08.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.180+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc980ab4d0 con 0x7fcc9813ce80 2023-12-12T09:53:08.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.180+0000 7fcc8e7fc640 1 -- 172.21.15.72:0/2964481235 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc88006930 con 0x7fcc9813ce80 2023-12-12T09:53:08.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.181+0000 7fcc8e7fc640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc7c06ae30 0x7fcc7c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.181+0000 7fcc8e7fc640 1 -- 172.21.15.72:0/2964481235 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcc88089710 con 0x7fcc9813ce80 2023-12-12T09:53:08.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.182+0000 7fcc9df26640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc7c06ae30 0x7fcc7c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.182+0000 7fcc9df26640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc7c06ae30 0x7fcc7c06d2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcc9815cd60 tx=0x7fcc94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:08.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.186+0000 7fcc8e7fc640 1 -- 172.21.15.72:0/2964481235 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc980ab4d0 con 0x7fcc9813ce80 2023-12-12T09:53:08.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.373+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcc980ab4d0 con 0x7fcc9813ce80 2023-12-12T09:53:08.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.374+0000 7fcc8e7fc640 1 -- 172.21.15.72:0/2964481235 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcc980ab4d0 con 0x7fcc9813ce80 2023-12-12T09:53:08.377 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:08.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc7c06ae30 msgr2=0x7fcc7c06d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc7c06ae30 0x7fcc7c06d2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcc9815cd60 tx=0x7fcc94009040 comp rx=0 tx=0).stop 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc9813ce80 msgr2=0x7fcc980a8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc9813ce80 0x7fcc980a8450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800d5c0 tx=0x7fcc8800da80 comp rx=0 tx=0).stop 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 shutdown_connections 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc7c06ae30 0x7fcc7c06d2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc980a8990 0x7fcc980a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc9813ce80 0x7fcc980a8450 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 --2- 172.21.15.72:0/2964481235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc84005180 0x7fcc9813d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 >> 172.21.15.72:0/2964481235 conn(0x7fcc980cb850 msgr2=0x7fcc980aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:08.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 shutdown_connections 2023-12-12T09:53:08.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.377+0000 7fcca018a640 1 -- 172.21.15.72:0/2964481235 wait complete. 2023-12-12T09:53:08.398 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:08.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.612+0000 7fee580e5640 1 Processor -- start 2023-12-12T09:53:08.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.612+0000 7fee580e5640 1 -- start start 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee580e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee501467c0 0x7fee500cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee580e5640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fee50146ba0 0x7fee500cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee580e5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee5006c7e0 0x7fee5006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee580e5640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee500ac860 con 0x7fee50146ba0 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee580e5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5006d510 con 0x7fee5006c7e0 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee580e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5006d7f0 con 0x7fee501467c0 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee55680640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fee50146ba0 0x7fee500cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee55680640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fee50146ba0 0x7fee500cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36194/0 (socket says 172.21.15.72:36194) 2023-12-12T09:53:08.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.614+0000 7fee55680640 1 -- 172.21.15.72:0/4157673777 learned_addr learned my addr 172.21.15.72:0/4157673777 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:08.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.615+0000 7fee55e81640 1 --2- 172.21.15.72:0/4157673777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee501467c0 0x7fee500cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.615+0000 7fee56682640 1 --2- 172.21.15.72:0/4157673777 >> v2:172.21.15.113:3300/0 conn(0x7fee5006c7e0 0x7fee5006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.615+0000 7fee55680640 1 -- 172.21.15.72:0/4157673777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee501467c0 msgr2=0x7fee500cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:08.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.615+0000 7fee55680640 1 --2- 172.21.15.72:0/4157673777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee501467c0 0x7fee500cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.615+0000 7fee55680640 1 -- 172.21.15.72:0/4157673777 >> v2:172.21.15.113:3300/0 conn(0x7fee5006c7e0 msgr2=0x7fee5006cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:08.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.615+0000 7fee55680640 1 --2- 172.21.15.72:0/4157673777 >> v2:172.21.15.113:3300/0 conn(0x7fee5006c7e0 0x7fee5006cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.615+0000 7fee55680640 1 -- 172.21.15.72:0/4157673777 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5006dad0 con 0x7fee50146ba0 2023-12-12T09:53:08.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.616+0000 7fee55680640 1 --2- 172.21.15.72:0/4157673777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee50146ba0 0x7fee500cce30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee500cb7b0 tx=0x7fee4402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e90db8d7645d13 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:08.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.616+0000 7fee54e7f640 1 -- 172.21.15.72:0/4157673777 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4402ca70 con 0x7fee50146ba0 2023-12-12T09:53:08.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.616+0000 7fee54e7f640 1 -- 172.21.15.72:0/4157673777 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4402cbd0 con 0x7fee50146ba0 2023-12-12T09:53:08.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.616+0000 7fee54e7f640 1 -- 172.21.15.72:0/4157673777 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee440343f0 con 0x7fee50146ba0 2023-12-12T09:53:08.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 -- 172.21.15.72:0/4157673777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee50146ba0 msgr2=0x7fee500cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 --2- 172.21.15.72:0/4157673777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee50146ba0 0x7fee500cce30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee500cb7b0 tx=0x7fee4402bed0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 -- 172.21.15.72:0/4157673777 shutdown_connections 2023-12-12T09:53:08.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 --2- 172.21.15.72:0/4157673777 >> v2:172.21.15.113:3300/0 conn(0x7fee5006c7e0 0x7fee5006cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 --2- 172.21.15.72:0/4157673777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee50146ba0 0x7fee500cce30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 --2- 172.21.15.72:0/4157673777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee501467c0 0x7fee500cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 -- 172.21.15.72:0/4157673777 >> 172.21.15.72:0/4157673777 conn(0x7fee500b0a90 msgr2=0x7fee500b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:08.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 -- 172.21.15.72:0/4157673777 shutdown_connections 2023-12-12T09:53:08.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.617+0000 7fee580e5640 1 -- 172.21.15.72:0/4157673777 wait complete. 2023-12-12T09:53:08.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.619+0000 7fee580e5640 1 Processor -- start 2023-12-12T09:53:08.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.619+0000 7fee580e5640 1 -- start start 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.620+0000 7fee580e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee5006c7e0 0x7fee50158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.620+0000 7fee580e5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee501467c0 0x7fee50130fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.620+0000 7fee580e5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee50146ba0 0x7fee501314f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.620+0000 7fee580e5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee501323b0 con 0x7fee501467c0 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.620+0000 7fee580e5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee501450b0 con 0x7fee50146ba0 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.620+0000 7fee580e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee501e96b0 con 0x7fee5006c7e0 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.620+0000 7fee55680640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee501467c0 0x7fee50130fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55680640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee501467c0 0x7fee50130fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36198/0 (socket says 172.21.15.72:36198) 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55680640 1 -- 172.21.15.72:0/3259549250 learned_addr learned my addr 172.21.15.72:0/3259549250 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55e81640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee5006c7e0 0x7fee50158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee56682640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee50146ba0 0x7fee501314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55680640 1 -- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee5006c7e0 msgr2=0x7fee50158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55680640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee5006c7e0 0x7fee50158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55680640 1 -- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee50146ba0 msgr2=0x7fee501314f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55680640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee50146ba0 0x7fee501314f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55680640 1 -- 172.21.15.72:0/3259549250 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee44009b30 con 0x7fee501467c0 2023-12-12T09:53:08.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee56682640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee50146ba0 0x7fee501314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:08.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.621+0000 7fee55680640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee501467c0 0x7fee50130fb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee44000f80 tx=0x7fee44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:08.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.622+0000 7fee427fc640 1 -- 172.21.15.72:0/3259549250 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44040670 con 0x7fee501467c0 2023-12-12T09:53:08.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.622+0000 7fee427fc640 1 -- 172.21.15.72:0/3259549250 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee440407f0 con 0x7fee501467c0 2023-12-12T09:53:08.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.622+0000 7fee427fc640 1 -- 172.21.15.72:0/3259549250 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44040ba0 con 0x7fee501467c0 2023-12-12T09:53:08.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.622+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee50131d30 con 0x7fee501467c0 2023-12-12T09:53:08.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.622+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee50131fd0 con 0x7fee501467c0 2023-12-12T09:53:08.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.623+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee501324f0 con 0x7fee501467c0 2023-12-12T09:53:08.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.624+0000 7fee427fc640 1 -- 172.21.15.72:0/3259549250 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee44040d00 con 0x7fee501467c0 2023-12-12T09:53:08.626 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.625+0000 7fee427fc640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee3406ae40 0x7fee3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:08.627 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.625+0000 7fee427fc640 1 -- 172.21.15.72:0/3259549250 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fee440acd60 con 0x7fee501467c0 2023-12-12T09:53:08.627 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.625+0000 7fee55e81640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee3406ae40 0x7fee3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:08.627 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.626+0000 7fee55e81640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee3406ae40 0x7fee3406d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006fd0 tx=0x7fee4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:08.631 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.630+0000 7fee427fc640 1 -- 172.21.15.72:0/3259549250 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee501324f0 con 0x7fee501467c0 2023-12-12T09:53:08.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.825+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee501324f0 con 0x7fee501467c0 2023-12-12T09:53:08.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.827+0000 7fee427fc640 1 -- 172.21.15.72:0/3259549250 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7fee501324f0 con 0x7fee501467c0 2023-12-12T09:53:08.829 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:08.829 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee3406ae40 msgr2=0x7fee3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee3406ae40 0x7fee3406d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006fd0 tx=0x7fee4c008040 comp rx=0 tx=0).stop 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee501467c0 msgr2=0x7fee50130fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee501467c0 0x7fee50130fb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee44000f80 tx=0x7fee44004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 shutdown_connections 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee3406ae40 0x7fee3406d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee50146ba0 0x7fee501314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee501467c0 0x7fee50130fb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 --2- 172.21.15.72:0/3259549250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee5006c7e0 0x7fee50158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 >> 172.21.15.72:0/3259549250 conn(0x7fee500b0a90 msgr2=0x7fee500b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 shutdown_connections 2023-12-12T09:53:08.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:08.830+0000 7fee580e5640 1 -- 172.21.15.72:0/3259549250 wait complete. 2023-12-12T09:53:08.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:08.852 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:09.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.063+0000 7fe02ff41640 1 Processor -- start 2023-12-12T09:53:09.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.063+0000 7fe02ff41640 1 -- start start 2023-12-12T09:53:09.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.065+0000 7fe02ff41640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe02815a1e0 0x7fe02806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.065+0000 7fe02ff41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 0x7fe02815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.065+0000 7fe02ff41640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0280cdaa0 con 0x7fe0280ae300 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.065+0000 7fe02ff41640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0280cdd80 con 0x7fe02815a1e0 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.065+0000 7fe02ff41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280ce060 con 0x7fe02806ab70 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.065+0000 7fe02dcdd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe02815a1e0 0x7fe02806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.065+0000 7fe02dcdd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe02815a1e0 0x7fe02806ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36206/0 (socket says 172.21.15.72:36206) 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.065+0000 7fe02dcdd640 1 -- 172.21.15.72:0/807296631 learned_addr learned my addr 172.21.15.72:0/807296631 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.066+0000 7fe02dcdd640 1 -- 172.21.15.72:0/807296631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 msgr2=0x7fe02815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.066+0000 7fe02e4de640 1 --2- 172.21.15.72:0/807296631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 0x7fe02815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.066+0000 7fe02dcdd640 1 --2- 172.21.15.72:0/807296631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 0x7fe02815dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.066+0000 7fe02dcdd640 1 -- 172.21.15.72:0/807296631 >> v1:172.21.15.72:6789/0 conn(0x7fe0280ae300 legacy=0x7fe02806a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.066+0000 7fe02dcdd640 1 -- 172.21.15.72:0/807296631 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0280ce340 con 0x7fe02815a1e0 2023-12-12T09:53:09.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.066+0000 7fe02ccdb640 1 -- 172.21.15.72:0/807296631 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524330817 0 0) 0x7fe0280cdaa0 con 0x7fe0280ae300 2023-12-12T09:53:09.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.067+0000 7fe02dcdd640 1 --2- 172.21.15.72:0/807296631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe02806ca10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe024009870 tx=0x7fe02402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9959d4e74d4cc4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.067+0000 7fe02ccdb640 1 -- 172.21.15.72:0/807296631 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024031640 con 0x7fe02815a1e0 2023-12-12T09:53:09.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.067+0000 7fe02ccdb640 1 -- 172.21.15.72:0/807296631 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0240317a0 con 0x7fe02815a1e0 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.067+0000 7fe02ccdb640 1 -- 172.21.15.72:0/807296631 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024031b30 con 0x7fe02815a1e0 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.068+0000 7fe02ff41640 1 -- 172.21.15.72:0/807296631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 msgr2=0x7fe02806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.068+0000 7fe02ff41640 1 --2- 172.21.15.72:0/807296631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe02806ca10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe024009870 tx=0x7fe02402b8f0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.068+0000 7fe02ff41640 1 -- 172.21.15.72:0/807296631 shutdown_connections 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.068+0000 7fe02ff41640 1 --2- 172.21.15.72:0/807296631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 0x7fe02815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.068+0000 7fe02ff41640 1 --2- 172.21.15.72:0/807296631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe02806ca10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.068+0000 7fe02ff41640 1 -- 172.21.15.72:0/807296631 >> 172.21.15.72:0/807296631 conn(0x7fe0280cee70 msgr2=0x7fe0280cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.068+0000 7fe02ff41640 1 -- 172.21.15.72:0/807296631 shutdown_connections 2023-12-12T09:53:09.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.068+0000 7fe02ff41640 1 -- 172.21.15.72:0/807296631 wait complete. 2023-12-12T09:53:09.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.070+0000 7fe02ff41640 1 Processor -- start 2023-12-12T09:53:09.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.070+0000 7fe02ff41640 1 -- start start 2023-12-12T09:53:09.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02ff41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 0x7fe02815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02ff41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0280ae300 0x7fe0280999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02ff41640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe028155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02ff41640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280b20e0 con 0x7fe02815a1e0 2023-12-12T09:53:09.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02ff41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe028099f30 con 0x7fe0280ae300 2023-12-12T09:53:09.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02ff41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02809a1e0 con 0x7fe02806ab70 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02e4de640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe028155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02e4de640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe028155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36218/0 (socket says 172.21.15.72:36218) 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02e4de640 1 -- 172.21.15.72:0/2471432977 learned_addr learned my addr 172.21.15.72:0/2471432977 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.071+0000 7fe02dcdd640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 0x7fe02815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe02e4de640 1 -- 172.21.15.72:0/2471432977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 msgr2=0x7fe02815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe02e4de640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 0x7fe02815c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe02e4de640 1 -- 172.21.15.72:0/2471432977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0280ae300 msgr2=0x7fe0280999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe02e4de640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0280ae300 0x7fe0280999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe02e4de640 1 -- 172.21.15.72:0/2471432977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0240094d0 con 0x7fe02815a1e0 2023-12-12T09:53:09.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe02e4de640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe028155ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe02000cd20 tx=0x7fe02000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe01e7fc640 1 -- 172.21.15.72:0/2471432977 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020003b90 con 0x7fe02815a1e0 2023-12-12T09:53:09.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe01e7fc640 1 -- 172.21.15.72:0/2471432977 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020003cf0 con 0x7fe02815a1e0 2023-12-12T09:53:09.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe01e7fc640 1 -- 172.21.15.72:0/2471432977 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020004240 con 0x7fe02815a1e0 2023-12-12T09:53:09.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02809a4c0 con 0x7fe02815a1e0 2023-12-12T09:53:09.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.072+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02809aa80 con 0x7fe02815a1e0 2023-12-12T09:53:09.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.073+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0280b1eb0 con 0x7fe02815a1e0 2023-12-12T09:53:09.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.074+0000 7fe01e7fc640 1 -- 172.21.15.72:0/2471432977 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe0200043a0 con 0x7fe02815a1e0 2023-12-12T09:53:09.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.076+0000 7fe01e7fc640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe00c06ae40 0x7fe00c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.076+0000 7fe01e7fc640 1 -- 172.21.15.72:0/2471432977 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe020012070 con 0x7fe02815a1e0 2023-12-12T09:53:09.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.076+0000 7fe02dcdd640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe00c06ae40 0x7fe00c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.077+0000 7fe02dcdd640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe00c06ae40 0x7fe00c06d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe024036000 tx=0x7fe02403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.080+0000 7fe01e7fc640 1 -- 172.21.15.72:0/2471432977 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0280b1eb0 con 0x7fe02815a1e0 2023-12-12T09:53:09.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.269+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0280b1eb0 con 0x7fe02815a1e0 2023-12-12T09:53:09.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.270+0000 7fe01e7fc640 1 -- 172.21.15.72:0/2471432977 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0280b1eb0 con 0x7fe02815a1e0 2023-12-12T09:53:09.272 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:09.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.272+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe00c06ae40 msgr2=0x7fe00c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.272+0000 7fe02ff41640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe00c06ae40 0x7fe00c06d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe024036000 tx=0x7fe02403c040 comp rx=0 tx=0).stop 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.272+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 msgr2=0x7fe028155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.272+0000 7fe02ff41640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe028155ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe02000cd20 tx=0x7fe02000a660 comp rx=0 tx=0).stop 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.273+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 shutdown_connections 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.273+0000 7fe02ff41640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe00c06ae40 0x7fe00c06d300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.273+0000 7fe02ff41640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe02815a1e0 0x7fe028155ee0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.273+0000 7fe02ff41640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0280ae300 0x7fe0280999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.273+0000 7fe02ff41640 1 --2- 172.21.15.72:0/2471432977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe02806ab70 0x7fe02815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.273+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 >> 172.21.15.72:0/2471432977 conn(0x7fe0280cee70 msgr2=0x7fe028071950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.273+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 shutdown_connections 2023-12-12T09:53:09.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.273+0000 7fe02ff41640 1 -- 172.21.15.72:0/2471432977 wait complete. 2023-12-12T09:53:09.295 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:09.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.518+0000 7fba44231640 1 Processor -- start 2023-12-12T09:53:09.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.519+0000 7fba44231640 1 -- start start 2023-12-12T09:53:09.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.520+0000 7fba44231640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fba3c15d4e0 0x7fba3c0ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.520+0000 7fba44231640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3c0cad40 con 0x7fba3c071a90 2023-12-12T09:53:09.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.520+0000 7fba44231640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3c06d870 con 0x7fba3c0ca8e0 2023-12-12T09:53:09.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.520+0000 7fba44231640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c071ee0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.520+0000 7fba41fcd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fba3c15d4e0 0x7fba3c0ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.521+0000 7fba427ce640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fba3c071a90 0x7fba3c06dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39974/0 (socket says 172.21.15.72:39974) 2023-12-12T09:53:09.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.521+0000 7fba427ce640 1 -- 172.21.15.72:0/4023818270 learned_addr learned my addr 172.21.15.72:0/4023818270 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:09.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.521+0000 7fba41fcd640 1 --2- 172.21.15.72:0/4023818270 >> v2:172.21.15.72:3300/0 conn(0x7fba3c15d4e0 0x7fba3c0ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36220/0 (socket says 172.21.15.72:36220) 2023-12-12T09:53:09.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.521+0000 7fba41fcd640 1 -- 172.21.15.72:0/4023818270 >> v1:172.21.15.113:6789/0 conn(0x7fba3c0ca8e0 legacy=0x7fba3c071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:09.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.521+0000 7fba41fcd640 1 -- 172.21.15.72:0/4023818270 >> v1:172.21.15.72:6789/0 conn(0x7fba3c071a90 legacy=0x7fba3c06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.521+0000 7fba41fcd640 1 -- 172.21.15.72:0/4023818270 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c06d260 con 0x7fba3c15d4e0 2023-12-12T09:53:09.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.521+0000 7fba40fcb640 1 -- 172.21.15.72:0/4023818270 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231476076 0 0) 0x7fba3c0cad40 con 0x7fba3c071a90 2023-12-12T09:53:09.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.522+0000 7fba41fcd640 1 --2- 172.21.15.72:0/4023818270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c0ca3a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009d90 tx=0x7fba2c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e44787749716056 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.522+0000 7fba40fcb640 1 -- 172.21.15.72:0/4023818270 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c02cbd0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.522+0000 7fba40fcb640 1 -- 172.21.15.72:0/4023818270 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c02cd30 con 0x7fba3c15d4e0 2023-12-12T09:53:09.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.522+0000 7fba40fcb640 1 -- 172.21.15.72:0/4023818270 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c02cea0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.523+0000 7fba44231640 1 -- 172.21.15.72:0/4023818270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 msgr2=0x7fba3c0ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.523+0000 7fba44231640 1 --2- 172.21.15.72:0/4023818270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c0ca3a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009d90 tx=0x7fba2c02be20 comp rx=0 tx=0).stop 2023-12-12T09:53:09.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.523+0000 7fba44231640 1 -- 172.21.15.72:0/4023818270 shutdown_connections 2023-12-12T09:53:09.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.523+0000 7fba44231640 1 --2- 172.21.15.72:0/4023818270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c0ca3a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.523+0000 7fba44231640 1 -- 172.21.15.72:0/4023818270 >> 172.21.15.72:0/4023818270 conn(0x7fba3c0cc7d0 msgr2=0x7fba3c0ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:09.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.523+0000 7fba44231640 1 -- 172.21.15.72:0/4023818270 shutdown_connections 2023-12-12T09:53:09.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.523+0000 7fba44231640 1 -- 172.21.15.72:0/4023818270 wait complete. 2023-12-12T09:53:09.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.525+0000 7fba44231640 1 Processor -- start 2023-12-12T09:53:09.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.525+0000 7fba44231640 1 -- start start 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.526+0000 7fba44231640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c071a90 0x7fba3c0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.526+0000 7fba44231640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c0ca8e0 0x7fba3c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.526+0000 7fba44231640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.526+0000 7fba44231640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c15f2f0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.526+0000 7fba44231640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c150730 con 0x7fba3c0ca8e0 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.526+0000 7fba44231640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c15f520 con 0x7fba3c071a90 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c1501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36232/0 (socket says 172.21.15.72:36232) 2023-12-12T09:53:09.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 -- 172.21.15.72:0/3923760520 learned_addr learned my addr 172.21.15.72:0/3923760520 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:09.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba41fcd640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c071a90 0x7fba3c0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba417cc640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c0ca8e0 0x7fba3c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 -- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c071a90 msgr2=0x7fba3c0c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c071a90 0x7fba3c0c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 -- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c0ca8e0 msgr2=0x7fba3c15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c0ca8e0 0x7fba3c15edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 -- 172.21.15.72:0/3923760520 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c009a70 con 0x7fba3c15d4e0 2023-12-12T09:53:09.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba417cc640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c0ca8e0 0x7fba3c15edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:09.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.527+0000 7fba427ce640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c1501f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c450 tx=0x7fba3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.528+0000 7fba3a7fc640 1 -- 172.21.15.72:0/3923760520 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba340074e0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.528+0000 7fba3a7fc640 1 -- 172.21.15.72:0/3923760520 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba34007660 con 0x7fba3c15d4e0 2023-12-12T09:53:09.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.528+0000 7fba3a7fc640 1 -- 172.21.15.72:0/3923760520 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34004a20 con 0x7fba3c15d4e0 2023-12-12T09:53:09.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.528+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c15f800 con 0x7fba3c15d4e0 2023-12-12T09:53:09.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.528+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c15fdc0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.529+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3c0cb7e0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.530+0000 7fba3a7fc640 1 -- 172.21.15.72:0/3923760520 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba34004b80 con 0x7fba3c15d4e0 2023-12-12T09:53:09.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.531+0000 7fba3a7fc640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fba2806ae40 0x7fba2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.531+0000 7fba3a7fc640 1 -- 172.21.15.72:0/3923760520 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fba34088bc0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.531+0000 7fba41fcd640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fba2806ae40 0x7fba2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.532+0000 7fba41fcd640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fba2806ae40 0x7fba2806d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba2c036000 tx=0x7fba2c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.535+0000 7fba3a7fc640 1 -- 172.21.15.72:0/3923760520 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba340538d0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.727+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba3c0745d0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.727+0000 7fba3a7fc640 1 -- 172.21.15.72:0/3923760520 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba340180a0 con 0x7fba3c15d4e0 2023-12-12T09:53:09.730 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fba2806ae40 msgr2=0x7fba2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fba2806ae40 0x7fba2806d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba2c036000 tx=0x7fba2c030000 comp rx=0 tx=0).stop 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 msgr2=0x7fba3c1501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c1501f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c450 tx=0x7fba3400c910 comp rx=0 tx=0).stop 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 shutdown_connections 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fba2806ae40 0x7fba2806d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba3c15d4e0 0x7fba3c1501f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c0ca8e0 0x7fba3c15edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 --2- 172.21.15.72:0/3923760520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c071a90 0x7fba3c0c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 >> 172.21.15.72:0/3923760520 conn(0x7fba3c0cc7d0 msgr2=0x7fba3c0cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:09.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 shutdown_connections 2023-12-12T09:53:09.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.730+0000 7fba44231640 1 -- 172.21.15.72:0/3923760520 wait complete. 2023-12-12T09:53:09.751 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:09.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.969+0000 7fbf90ce0640 1 Processor -- start 2023-12-12T09:53:09.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.969+0000 7fbf90ce0640 1 -- start start 2023-12-12T09:53:09.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.970+0000 7fbf90ce0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf8c15a1e0 0x7fbf8c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.970+0000 7fbf90ce0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 0x7fbf8c15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.970+0000 7fbf90ce0640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c0cdaa0 con 0x7fbf8c0ae300 2023-12-12T09:53:09.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.970+0000 7fbf90ce0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8c0cdd80 con 0x7fbf8c15a1e0 2023-12-12T09:53:09.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.970+0000 7fbf90ce0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0ce060 con 0x7fbf8c06ab70 2023-12-12T09:53:09.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf83fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fbf8c0ae300 0x7fbf8c06a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39986/0 (socket says 172.21.15.72:39986) 2023-12-12T09:53:09.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf83fff640 1 -- 172.21.15.72:0/2633615396 learned_addr learned my addr 172.21.15.72:0/2633615396 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf8a59c640 1 --2- 172.21.15.72:0/2633615396 >> v2:172.21.15.113:3300/0 conn(0x7fbf8c15a1e0 0x7fbf8c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf8ad9d640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 0x7fbf8c15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29848230 0 0) 0x7fbf8c0cdaa0 con 0x7fbf8c0ae300 2023-12-12T09:53:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf78003520 con 0x7fbf8c0ae300 2023-12-12T09:53:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4239712974 0 0) 0x7fbf78003520 con 0x7fbf8c0ae300 2023-12-12T09:53:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 msgr2=0x7fbf8c15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf89d9b640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 0x7fbf8c15dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 >> v2:172.21.15.113:3300/0 conn(0x7fbf8c15a1e0 msgr2=0x7fbf8c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.971+0000 7fbf89d9b640 1 --2- 172.21.15.72:0/2633615396 >> v2:172.21.15.113:3300/0 conn(0x7fbf8c15a1e0 0x7fbf8c06ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0ce340 con 0x7fbf8c0ae300 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fbf74001670 con 0x7fbf8c0ae300 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf8a59c640 1 --2- 172.21.15.72:0/2633615396 >> v2:172.21.15.113:3300/0 conn(0x7fbf8c15a1e0 0x7fbf8c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 >> v1:172.21.15.72:6789/0 conn(0x7fbf8c0ae300 legacy=0x7fbf8c06a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf78003970 0x7fbf78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf780047c0 0x7fbf78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 0x7fbf78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0cdd80 con 0x7fbf780047c0 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0ce060 con 0x7fbf78003970 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7800bb10 con 0x7fbf8c06ab70 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf83fff640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf780047c0 0x7fbf78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf8a59c640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf78003970 0x7fbf78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf8ad9d640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 0x7fbf78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf83fff640 1 -- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 msgr2=0x7fbf78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:09.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf83fff640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 0x7fbf78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf83fff640 1 -- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf78003970 msgr2=0x7fbf78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:09.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf83fff640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf78003970 0x7fbf78003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf83fff640 1 -- 172.21.15.72:0/2633615396 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78003520 con 0x7fbf780047c0 2023-12-12T09:53:09.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf8a59c640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf78003970 0x7fbf78003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:09.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.972+0000 7fbf8ad9d640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf8c06ab70 0x7fbf78005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:09.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf83fff640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf780047c0 0x7fbf78004c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400c210 tx=0x7fbf780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00db27b264dee8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74005ad0 con 0x7fbf780047c0 2023-12-12T09:53:09.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf74005c30 con 0x7fbf780047c0 2023-12-12T09:53:09.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf89d9b640 1 -- 172.21.15.72:0/2633615396 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7402c8b0 con 0x7fbf780047c0 2023-12-12T09:53:09.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf90ce0640 1 -- 172.21.15.72:0/2633615396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf780047c0 msgr2=0x7fbf78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf90ce0640 1 --2- 172.21.15.72:0/2633615396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf780047c0 0x7fbf78004c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400c210 tx=0x7fbf780057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf8ad9d640 1 -- 172.21.15.72:0/2633615396 reap_dead start 2023-12-12T09:53:09.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf90ce0640 1 -- 172.21.15.72:0/2633615396 shutdown_connections 2023-12-12T09:53:09.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.974+0000 7fbf90ce0640 1 -- 172.21.15.72:0/2633615396 >> 172.21.15.72:0/2633615396 conn(0x7fbf8c0cee70 msgr2=0x7fbf8c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:09.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.975+0000 7fbf90ce0640 1 -- 172.21.15.72:0/2633615396 shutdown_connections 2023-12-12T09:53:09.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.975+0000 7fbf90ce0640 1 -- 172.21.15.72:0/2633615396 wait complete. 2023-12-12T09:53:09.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.976+0000 7fbf90ce0640 1 Processor -- start 2023-12-12T09:53:09.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.976+0000 7fbf90ce0640 1 -- start start 2023-12-12T09:53:09.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.977+0000 7fbf90ce0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf780047c0 0x7fbf8c14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.977+0000 7fbf90ce0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf8c0acd70 0x7fbf8c0ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.977+0000 7fbf90ce0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c150260 0x7fbf8c1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.977+0000 7fbf90ce0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0ad710 con 0x7fbf8c150260 2023-12-12T09:53:09.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.977+0000 7fbf90ce0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c15b170 con 0x7fbf8c0acd70 2023-12-12T09:53:09.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.977+0000 7fbf90ce0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c15b450 con 0x7fbf780047c0 2023-12-12T09:53:09.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c150260 0x7fbf8c1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c150260 0x7fbf8c1506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36260/0 (socket says 172.21.15.72:36260) 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 -- 172.21.15.72:0/3742408743 learned_addr learned my addr 172.21.15.72:0/3742408743 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8a59c640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf780047c0 0x7fbf8c14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf83fff640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf8c0acd70 0x7fbf8c0ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 -- 172.21.15.72:0/3742408743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf780047c0 msgr2=0x7fbf8c14fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf780047c0 0x7fbf8c14fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 -- 172.21.15.72:0/3742408743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf8c0acd70 msgr2=0x7fbf8c0ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf8c0acd70 0x7fbf8c0ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 -- 172.21.15.72:0/3742408743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c15b730 con 0x7fbf8c150260 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.978+0000 7fbf8ad9d640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c150260 0x7fbf8c1506e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c06e660 tx=0x7fbf8400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.979+0000 7fbf82ffd640 1 -- 172.21.15.72:0/3742408743 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400fd80 con 0x7fbf8c150260 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.979+0000 7fbf82ffd640 1 -- 172.21.15.72:0/3742408743 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf840143f0 con 0x7fbf8c150260 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.979+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c15ba10 con 0x7fbf8c150260 2023-12-12T09:53:09.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.979+0000 7fbf82ffd640 1 -- 172.21.15.72:0/3742408743 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400fee0 con 0x7fbf8c150260 2023-12-12T09:53:09.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.979+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c150000 con 0x7fbf8c150260 2023-12-12T09:53:09.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.980+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8c15bf30 con 0x7fbf8c150260 2023-12-12T09:53:09.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.981+0000 7fbf82ffd640 1 -- 172.21.15.72:0/3742408743 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf840147a0 con 0x7fbf8c150260 2023-12-12T09:53:09.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.982+0000 7fbf82ffd640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbf6c06ae70 0x7fbf6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:09.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.982+0000 7fbf82ffd640 1 -- 172.21.15.72:0/3742408743 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbf84089ec0 con 0x7fbf8c150260 2023-12-12T09:53:09.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.982+0000 7fbf8a59c640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbf6c06ae70 0x7fbf6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:09.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.983+0000 7fbf8a59c640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbf6c06ae70 0x7fbf6c06d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c006fd0 tx=0x7fbf7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:09.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:09.986+0000 7fbf82ffd640 1 -- 172.21.15.72:0/3742408743 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf8c15bf30 con 0x7fbf8c150260 2023-12-12T09:53:10.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.175+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf8c15bf30 con 0x7fbf8c150260 2023-12-12T09:53:10.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.176+0000 7fbf82ffd640 1 -- 172.21.15.72:0/3742408743 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf8c15bf30 con 0x7fbf8c150260 2023-12-12T09:53:10.179 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbf6c06ae70 msgr2=0x7fbf6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbf6c06ae70 0x7fbf6c06d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c006fd0 tx=0x7fbf7c008040 comp rx=0 tx=0).stop 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c150260 msgr2=0x7fbf8c1506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c150260 0x7fbf8c1506e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c06e660 tx=0x7fbf8400f520 comp rx=0 tx=0).stop 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 shutdown_connections 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbf6c06ae70 0x7fbf6c06d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c150260 0x7fbf8c1506e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf8c0acd70 0x7fbf8c0ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 --2- 172.21.15.72:0/3742408743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf780047c0 0x7fbf8c14fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 >> 172.21.15.72:0/3742408743 conn(0x7fbf8c0cee70 msgr2=0x7fbf8c06d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:10.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 shutdown_connections 2023-12-12T09:53:10.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.179+0000 7fbf90ce0640 1 -- 172.21.15.72:0/3742408743 wait complete. 2023-12-12T09:53:10.200 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:10.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.407+0000 7fe2f821e640 1 Processor -- start 2023-12-12T09:53:10.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.407+0000 7fe2f821e640 1 -- start start 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f821e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe2f006db50 0x7fe2f0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f821e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe2f0141790 0x7fe2f00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f821e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f015b780 con 0x7fe2f006d770 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f821e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f0141c20 con 0x7fe2f0141790 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f821e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f00b2510 con 0x7fe2f006db50 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f67bb640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe2f0141790 0x7fe2f00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f67bb640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe2f0141790 0x7fe2f00cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36268/0 (socket says 172.21.15.72:36268) 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f5fba640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fe2f006d770 0x7fe2f00b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40002/0 (socket says 172.21.15.72:40002) 2023-12-12T09:53:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f67bb640 1 -- 172.21.15.72:0/3697465894 learned_addr learned my addr 172.21.15.72:0/3697465894 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f67bb640 1 -- 172.21.15.72:0/3697465894 >> v2:172.21.15.113:3300/0 conn(0x7fe2f006db50 msgr2=0x7fe2f0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.409+0000 7fe2f57b9640 1 --2- 172.21.15.72:0/3697465894 >> v2:172.21.15.113:3300/0 conn(0x7fe2f006db50 0x7fe2f0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.410+0000 7fe2f67bb640 1 --2- 172.21.15.72:0/3697465894 >> v2:172.21.15.113:3300/0 conn(0x7fe2f006db50 0x7fe2f0141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.410+0000 7fe2f67bb640 1 -- 172.21.15.72:0/3697465894 >> v1:172.21.15.72:6789/0 conn(0x7fe2f006d770 legacy=0x7fe2f00b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.410+0000 7fe2f67bb640 1 -- 172.21.15.72:0/3697465894 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f015b9d0 con 0x7fe2f0141790 2023-12-12T09:53:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.410+0000 7fe2f4fb8640 1 -- 172.21.15.72:0/3697465894 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232130781 0 0) 0x7fe2f015b780 con 0x7fe2f006d770 2023-12-12T09:53:10.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.411+0000 7fe2f67bb640 1 --2- 172.21.15.72:0/3697465894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f0141790 0x7fe2f00cce00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0067c70 tx=0x7fe2e802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd4d5a3e8db0a6f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:10.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.411+0000 7fe2f4fb8640 1 -- 172.21.15.72:0/3697465894 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e802cb10 con 0x7fe2f0141790 2023-12-12T09:53:10.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.411+0000 7fe2f4fb8640 1 -- 172.21.15.72:0/3697465894 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e802cc70 con 0x7fe2f0141790 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.411+0000 7fe2f4fb8640 1 -- 172.21.15.72:0/3697465894 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80343f0 con 0x7fe2f0141790 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.411+0000 7fe2f821e640 1 -- 172.21.15.72:0/3697465894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f0141790 msgr2=0x7fe2f00cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.411+0000 7fe2f821e640 1 --2- 172.21.15.72:0/3697465894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f0141790 0x7fe2f00cce00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0067c70 tx=0x7fe2e802bf70 comp rx=0 tx=0).stop 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.412+0000 7fe2f821e640 1 -- 172.21.15.72:0/3697465894 shutdown_connections 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.412+0000 7fe2f821e640 1 --2- 172.21.15.72:0/3697465894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f0141790 0x7fe2f00cce00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.412+0000 7fe2f821e640 1 --2- 172.21.15.72:0/3697465894 >> v2:172.21.15.113:3300/0 conn(0x7fe2f006db50 0x7fe2f0141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.412+0000 7fe2f821e640 1 -- 172.21.15.72:0/3697465894 >> 172.21.15.72:0/3697465894 conn(0x7fe2f00cb850 msgr2=0x7fe2f00cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.412+0000 7fe2f821e640 1 -- 172.21.15.72:0/3697465894 shutdown_connections 2023-12-12T09:53:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.412+0000 7fe2f821e640 1 -- 172.21.15.72:0/3697465894 wait complete. 2023-12-12T09:53:10.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.413+0000 7fe2f821e640 1 Processor -- start 2023-12-12T09:53:10.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.414+0000 7fe2f821e640 1 -- start start 2023-12-12T09:53:10.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.414+0000 7fe2f821e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2f006d770 0x7fe2f00c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.414+0000 7fe2f821e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f006db50 0x7fe2f00ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.414+0000 7fe2f821e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f0141790 0x7fe2f014c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.414+0000 7fe2f821e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f014da10 con 0x7fe2f006db50 2023-12-12T09:53:10.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.414+0000 7fe2f821e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f014bc20 con 0x7fe2f0141790 2023-12-12T09:53:10.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f821e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00c42b0 con 0x7fe2f006d770 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f006db50 0x7fe2f00ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f006db50 0x7fe2f00ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36272/0 (socket says 172.21.15.72:36272) 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 -- 172.21.15.72:0/1957768299 learned_addr learned my addr 172.21.15.72:0/1957768299 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 -- 172.21.15.72:0/1957768299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2f006d770 msgr2=0x7fe2f00c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2f006d770 0x7fe2f00c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 -- 172.21.15.72:0/1957768299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f0141790 msgr2=0x7fe2f014c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f0141790 0x7fe2f014c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 -- 172.21.15.72:0/1957768299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8009b30 con 0x7fe2f006db50 2023-12-12T09:53:10.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.415+0000 7fe2f57b9640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f006db50 0x7fe2f00ada00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c7f0 tx=0x7fe2e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:10.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.416+0000 7fe2e67fc640 1 -- 172.21.15.72:0/1957768299 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0004220 con 0x7fe2f006db50 2023-12-12T09:53:10.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.416+0000 7fe2e67fc640 1 -- 172.21.15.72:0/1957768299 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e00043a0 con 0x7fe2f006db50 2023-12-12T09:53:10.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.416+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f014b010 con 0x7fe2f006db50 2023-12-12T09:53:10.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.416+0000 7fe2e67fc640 1 -- 172.21.15.72:0/1957768299 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0007860 con 0x7fe2f006db50 2023-12-12T09:53:10.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.416+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f014b5d0 con 0x7fe2f006db50 2023-12-12T09:53:10.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.417+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f00745d0 con 0x7fe2f006db50 2023-12-12T09:53:10.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.418+0000 7fe2e67fc640 1 -- 172.21.15.72:0/1957768299 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe2e0007bd0 con 0x7fe2f006db50 2023-12-12T09:53:10.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.419+0000 7fe2e67fc640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe2d406ae70 0x7fe2d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.419+0000 7fe2e67fc640 1 -- 172.21.15.72:0/1957768299 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe2e00899a0 con 0x7fe2f006db50 2023-12-12T09:53:10.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.419+0000 7fe2f5fba640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe2d406ae70 0x7fe2d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.420+0000 7fe2f5fba640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe2d406ae70 0x7fe2d406d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec007fd0 tx=0x7fe2ec007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:10.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.423+0000 7fe2e67fc640 1 -- 172.21.15.72:0/1957768299 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2e0054680 con 0x7fe2f006db50 2023-12-12T09:53:10.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.602+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe2f014bd60 con 0x7fe2f006db50 2023-12-12T09:53:10.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.603+0000 7fe2e67fc640 1 -- 172.21.15.72:0/1957768299 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe2e001e0a0 con 0x7fe2f006db50 2023-12-12T09:53:10.605 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.605+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe2d406ae70 msgr2=0x7fe2d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.605+0000 7fe2f821e640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe2d406ae70 0x7fe2d406d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec007fd0 tx=0x7fe2ec007920 comp rx=0 tx=0).stop 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.605+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f006db50 msgr2=0x7fe2f00ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.605+0000 7fe2f821e640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f006db50 0x7fe2f00ada00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c7f0 tx=0x7fe2e000ccb0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.606+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 shutdown_connections 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.606+0000 7fe2f821e640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe2d406ae70 0x7fe2d406d330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.606+0000 7fe2f821e640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f0141790 0x7fe2f014c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.606+0000 7fe2f821e640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2f006db50 0x7fe2f00ada00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.606+0000 7fe2f821e640 1 --2- 172.21.15.72:0/1957768299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2f006d770 0x7fe2f00c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.606+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 >> 172.21.15.72:0/1957768299 conn(0x7fe2f00cb850 msgr2=0x7fe2f00ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.606+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 shutdown_connections 2023-12-12T09:53:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.606+0000 7fe2f821e640 1 -- 172.21.15.72:0/1957768299 wait complete. 2023-12-12T09:53:10.625 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:10.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.824+0000 7f5166b1d640 1 Processor -- start 2023-12-12T09:53:10.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.824+0000 7f5166b1d640 1 -- start start 2023-12-12T09:53:10.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.825+0000 7f5166b1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516006c410 0x7f5160166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.825+0000 7f5166b1d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51600b3bb0 con 0x7f5160072180 2023-12-12T09:53:10.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.825+0000 7f5166b1d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51600b4e60 con 0x7f5160143e00 2023-12-12T09:53:10.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.825+0000 7f5166b1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51601441e0 con 0x7f516006c410 2023-12-12T09:53:10.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.826+0000 7f515ffff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f5160072180 0x7f51600b4750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40012/0 (socket says 172.21.15.72:40012) 2023-12-12T09:53:10.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.826+0000 7f515ffff640 1 -- 172.21.15.72:0/158591640 learned_addr learned my addr 172.21.15.72:0/158591640 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:10.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.826+0000 7f51650ba640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516006c410 0x7f5160166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.826+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514037854 0 0) 0x7f51600b3bb0 con 0x7f5160072180 2023-12-12T09:53:10.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.826+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5150003520 con 0x7f5160072180 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 683337571 0 0) 0x7f5150003520 con 0x7f5160072180 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516006c410 msgr2=0x7f5160166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516006c410 0x7f5160166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 >> v1:172.21.15.113:6789/0 conn(0x7f5160143e00 legacy=0x7f51600b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160072680 con 0x7f5160072180 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f514c001670 con 0x7f5160072180 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 >> v1:172.21.15.72:6789/0 conn(0x7f5160072180 legacy=0x7f51600b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51500039a0 0x7f5150003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516006c410 0x7f51500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5150004d00 0x7f5150005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51601441e0 con 0x7f51500039a0 2023-12-12T09:53:10.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515000ba60 con 0x7f5150004d00 2023-12-12T09:53:10.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515000bc20 con 0x7f516006c410 2023-12-12T09:53:10.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f51648b9640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51500039a0 0x7f5150003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f51648b9640 1 -- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516006c410 msgr2=0x7f51500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:10.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f51648b9640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516006c410 0x7f51500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f51648b9640 1 -- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5150004d00 msgr2=0x7f5150005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:10.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f51648b9640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5150004d00 0x7f5150005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.827+0000 7f51648b9640 1 -- 172.21.15.72:0/158591640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150003520 con 0x7f51500039a0 2023-12-12T09:53:10.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.828+0000 7f51648b9640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51500039a0 0x7f5150003dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5154001720 tx=0x7f51500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71128f432edbd26 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:10.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51540052d0 con 0x7f51500039a0 2023-12-12T09:53:10.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515402e050 con 0x7f51500039a0 2023-12-12T09:53:10.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f515f7fe640 1 -- 172.21.15.72:0/158591640 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51540035f0 con 0x7f51500039a0 2023-12-12T09:53:10.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f5166b1d640 1 -- 172.21.15.72:0/158591640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51500039a0 msgr2=0x7f5150003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f5166b1d640 1 --2- 172.21.15.72:0/158591640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51500039a0 0x7f5150003dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5154001720 tx=0x7f51500057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f51650ba640 1 -- 172.21.15.72:0/158591640 reap_dead start 2023-12-12T09:53:10.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f5166b1d640 1 -- 172.21.15.72:0/158591640 shutdown_connections 2023-12-12T09:53:10.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f5166b1d640 1 -- 172.21.15.72:0/158591640 >> 172.21.15.72:0/158591640 conn(0x7f51600b0a90 msgr2=0x7f51600b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:10.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f5166b1d640 1 -- 172.21.15.72:0/158591640 shutdown_connections 2023-12-12T09:53:10.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.829+0000 7f5166b1d640 1 -- 172.21.15.72:0/158591640 wait complete. 2023-12-12T09:53:10.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.831+0000 7f5166b1d640 1 Processor -- start 2023-12-12T09:53:10.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.831+0000 7f5166b1d640 1 -- start start 2023-12-12T09:53:10.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.832+0000 7f5166b1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51500039a0 0x7f516013b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.832+0000 7f5166b1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516013e270 0x7f516013e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.832+0000 7f5166b1d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f516015c420 0x7f516015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.832+0000 7f5166b1d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516013ec10 con 0x7f516015c420 2023-12-12T09:53:10.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.832+0000 7f5166b1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5160141490 con 0x7f51500039a0 2023-12-12T09:53:10.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.832+0000 7f5166b1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516013bec0 con 0x7f516013e270 2023-12-12T09:53:10.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.832+0000 7f51650ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f516015c420 0x7f516015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.832+0000 7f51650ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f516015c420 0x7f516015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36282/0 (socket says 172.21.15.72:36282) 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f51650ba640 1 -- 172.21.15.72:0/1419839329 learned_addr learned my addr 172.21.15.72:0/1419839329 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f51648b9640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51500039a0 0x7f516013b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f515ffff640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516013e270 0x7f516013e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f51650ba640 1 -- 172.21.15.72:0/1419839329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516013e270 msgr2=0x7f516013e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f51650ba640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516013e270 0x7f516013e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f51650ba640 1 -- 172.21.15.72:0/1419839329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51500039a0 msgr2=0x7f516013b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f51650ba640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51500039a0 0x7f516013b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f51650ba640 1 -- 172.21.15.72:0/1419839329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516015cde0 con 0x7f516015c420 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f51650ba640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f516015c420 0x7f516015c8a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f516006b420 tx=0x7f515800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f515d7fa640 1 -- 172.21.15.72:0/1419839329 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158010040 con 0x7f516015c420 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f515d7fa640 1 -- 172.21.15.72:0/1419839329 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515800bce0 con 0x7f516015c420 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f515d7fa640 1 -- 172.21.15.72:0/1419839329 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158014640 con 0x7f516015c420 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.833+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516013fd50 con 0x7f516015c420 2023-12-12T09:53:10.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.834+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5160140310 con 0x7f516015c420 2023-12-12T09:53:10.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.835+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51600745d0 con 0x7f516015c420 2023-12-12T09:53:10.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.836+0000 7f515d7fa640 1 -- 172.21.15.72:0/1419839329 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51580147a0 con 0x7f516015c420 2023-12-12T09:53:10.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.836+0000 7f515d7fa640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f514406ae70 0x7f514406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:10.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.837+0000 7f515d7fa640 1 -- 172.21.15.72:0/1419839329 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5158089fc0 con 0x7f516015c420 2023-12-12T09:53:10.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.837+0000 7f51648b9640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f514406ae70 0x7f514406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:10.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.837+0000 7f51648b9640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f514406ae70 0x7f514406d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5154033000 tx=0x7f5154009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:10.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:10.841+0000 7f515d7fa640 1 -- 172.21.15.72:0/1419839329 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5158054ca0 con 0x7f516015c420 2023-12-12T09:53:11.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.036+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f516006e020 con 0x7f516015c420 2023-12-12T09:53:11.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.038+0000 7f515d7fa640 1 -- 172.21.15.72:0/1419839329 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7f5158018090 con 0x7f516015c420 2023-12-12T09:53:11.040 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:11.040 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f514406ae70 msgr2=0x7f514406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f514406ae70 0x7f514406d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5154033000 tx=0x7f5154009030 comp rx=0 tx=0).stop 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f516015c420 msgr2=0x7f516015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f516015c420 0x7f516015c8a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f516006b420 tx=0x7f515800b480 comp rx=0 tx=0).stop 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 shutdown_connections 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f514406ae70 0x7f514406d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f516015c420 0x7f516015c8a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f516013e270 0x7f516013e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 --2- 172.21.15.72:0/1419839329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51500039a0 0x7f516013b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 >> 172.21.15.72:0/1419839329 conn(0x7f51600b0a90 msgr2=0x7f51600b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:11.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 shutdown_connections 2023-12-12T09:53:11.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.041+0000 7f5166b1d640 1 -- 172.21.15.72:0/1419839329 wait complete. 2023-12-12T09:53:11.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:11.062 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:11.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.267+0000 7faed063d640 1 Processor -- start 2023-12-12T09:53:11.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.268+0000 7faed063d640 1 -- start start 2023-12-12T09:53:11.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.269+0000 7faed063d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faec806ab70 0x7faec806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.269+0000 7faed063d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faec80ae300 0x7faec815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.269+0000 7faed063d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec80cdaa0 con 0x7faec815a1e0 2023-12-12T09:53:11.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.269+0000 7faed063d640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec80cdd80 con 0x7faec806ab70 2023-12-12T09:53:11.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.269+0000 7faed063d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec80ce060 con 0x7faec80ae300 2023-12-12T09:53:11.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.269+0000 7faecdbd8640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faec806ab70 0x7faec806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.270+0000 7faecdbd8640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faec806ab70 0x7faec806ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36290/0 (socket says 172.21.15.72:36290) 2023-12-12T09:53:11.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.270+0000 7faecdbd8640 1 -- 172.21.15.72:0/1512096170 learned_addr learned my addr 172.21.15.72:0/1512096170 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:11.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.270+0000 7faecebda640 1 --2- 172.21.15.72:0/1512096170 >> v2:172.21.15.113:3300/0 conn(0x7faec80ae300 0x7faec815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.270+0000 7faecdbd8640 1 -- 172.21.15.72:0/1512096170 >> v2:172.21.15.113:3300/0 conn(0x7faec80ae300 msgr2=0x7faec815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:11.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.270+0000 7faecdbd8640 1 --2- 172.21.15.72:0/1512096170 >> v2:172.21.15.113:3300/0 conn(0x7faec80ae300 0x7faec815dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.270+0000 7faecdbd8640 1 -- 172.21.15.72:0/1512096170 >> v1:172.21.15.72:6789/0 conn(0x7faec815a1e0 legacy=0x7faec806a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.270+0000 7faecdbd8640 1 -- 172.21.15.72:0/1512096170 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec80ce340 con 0x7faec806ab70 2023-12-12T09:53:11.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.270+0000 7faecd3d7640 1 -- 172.21.15.72:0/1512096170 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034150454 0 0) 0x7faec80cdaa0 con 0x7faec815a1e0 2023-12-12T09:53:11.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.271+0000 7faecdbd8640 1 --2- 172.21.15.72:0/1512096170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec806ab70 0x7faec806ca10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faec80c5cf0 tx=0x7faeb802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce2826042327745 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:11.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faecd3d7640 1 -- 172.21.15.72:0/1512096170 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80317e0 con 0x7faec806ab70 2023-12-12T09:53:11.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faecd3d7640 1 -- 172.21.15.72:0/1512096170 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8031940 con 0x7faec806ab70 2023-12-12T09:53:11.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faecd3d7640 1 -- 172.21.15.72:0/1512096170 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8031cd0 con 0x7faec806ab70 2023-12-12T09:53:11.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faed063d640 1 -- 172.21.15.72:0/1512096170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec806ab70 msgr2=0x7faec806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faed063d640 1 --2- 172.21.15.72:0/1512096170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec806ab70 0x7faec806ca10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faec80c5cf0 tx=0x7faeb802b880 comp rx=0 tx=0).stop 2023-12-12T09:53:11.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faed063d640 1 -- 172.21.15.72:0/1512096170 shutdown_connections 2023-12-12T09:53:11.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faed063d640 1 --2- 172.21.15.72:0/1512096170 >> v2:172.21.15.113:3300/0 conn(0x7faec80ae300 0x7faec815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faed063d640 1 --2- 172.21.15.72:0/1512096170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec806ab70 0x7faec806ca10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faed063d640 1 -- 172.21.15.72:0/1512096170 >> 172.21.15.72:0/1512096170 conn(0x7faec80cee70 msgr2=0x7faec80cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:11.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faed063d640 1 -- 172.21.15.72:0/1512096170 shutdown_connections 2023-12-12T09:53:11.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.272+0000 7faed063d640 1 -- 172.21.15.72:0/1512096170 wait complete. 2023-12-12T09:53:11.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.274+0000 7faed063d640 1 Processor -- start 2023-12-12T09:53:11.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.274+0000 7faed063d640 1 -- start start 2023-12-12T09:53:11.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.275+0000 7faed063d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faec806ab70 0x7faec80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.275+0000 7faed063d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faec80ae300 0x7faec80b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.275+0000 7faed063d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec815a1e0 0x7faec809c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.275+0000 7faed063d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec8157690 con 0x7faec815a1e0 2023-12-12T09:53:11.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.275+0000 7faed063d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b4260 con 0x7faec80ae300 2023-12-12T09:53:11.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.275+0000 7faed063d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec80c3d90 con 0x7faec806ab70 2023-12-12T09:53:11.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec815a1e0 0x7faec809c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec815a1e0 0x7faec809c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36298/0 (socket says 172.21.15.72:36298) 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 -- 172.21.15.72:0/882760079 learned_addr learned my addr 172.21.15.72:0/882760079 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecdbd8640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faec80ae300 0x7faec80b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faece3d9640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faec806ab70 0x7faec80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 -- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faec806ab70 msgr2=0x7faec80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faec806ab70 0x7faec80c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 -- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faec80ae300 msgr2=0x7faec80b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faec80ae300 0x7faec80b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 -- 172.21.15.72:0/882760079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80094d0 con 0x7faec815a1e0 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faece3d9640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faec806ab70 0x7faec80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:11.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.276+0000 7faecebda640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec815a1e0 0x7faec809c410 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faec0007e40 tx=0x7faec000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:11.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.277+0000 7faebeffd640 1 -- 172.21.15.72:0/882760079 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0014620 con 0x7faec815a1e0 2023-12-12T09:53:11.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.277+0000 7faebeffd640 1 -- 172.21.15.72:0/882760079 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec000ee90 con 0x7faec815a1e0 2023-12-12T09:53:11.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.277+0000 7faebeffd640 1 -- 172.21.15.72:0/882760079 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00149e0 con 0x7faec815a1e0 2023-12-12T09:53:11.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.277+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec80c4070 con 0x7faec815a1e0 2023-12-12T09:53:11.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.277+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec80c4630 con 0x7faec815a1e0 2023-12-12T09:53:11.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.278+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec8074450 con 0x7faec815a1e0 2023-12-12T09:53:11.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.279+0000 7faebeffd640 1 -- 172.21.15.72:0/882760079 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faec0014b40 con 0x7faec815a1e0 2023-12-12T09:53:11.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.280+0000 7faebeffd640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7faeac06ae70 0x7faeac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.280+0000 7faebeffd640 1 -- 172.21.15.72:0/882760079 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faec008a390 con 0x7faec815a1e0 2023-12-12T09:53:11.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.280+0000 7faece3d9640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7faeac06ae70 0x7faeac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.280+0000 7faece3d9640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7faeac06ae70 0x7faeac06d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faec4000f60 tx=0x7faec4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:11.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.284+0000 7faebeffd640 1 -- 172.21.15.72:0/882760079 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faec0055070 con 0x7faec815a1e0 2023-12-12T09:53:11.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.473+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faec80c9b60 con 0x7faec815a1e0 2023-12-12T09:53:11.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.474+0000 7faebeffd640 1 -- 172.21.15.72:0/882760079 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7faec0058690 con 0x7faec815a1e0 2023-12-12T09:53:11.476 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7faeac06ae70 msgr2=0x7faeac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7faeac06ae70 0x7faeac06d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faec4000f60 tx=0x7faec4009040 comp rx=0 tx=0).stop 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec815a1e0 msgr2=0x7faec809c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec815a1e0 0x7faec809c410 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faec0007e40 tx=0x7faec000e4b0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 shutdown_connections 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7faeac06ae70 0x7faeac06d330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faec815a1e0 0x7faec809c410 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faec80ae300 0x7faec80b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 --2- 172.21.15.72:0/882760079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faec806ab70 0x7faec80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 >> 172.21.15.72:0/882760079 conn(0x7faec80cee70 msgr2=0x7faec806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 shutdown_connections 2023-12-12T09:53:11.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.477+0000 7faed063d640 1 -- 172.21.15.72:0/882760079 wait complete. 2023-12-12T09:53:11.498 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:11.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.704+0000 7fd3c8983640 1 Processor -- start 2023-12-12T09:53:11.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.704+0000 7fd3c8983640 1 -- start start 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3c8983640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 0x7fd3c40cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3c8983640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd3c41432b0 0x7fd3c406d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3c8983640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3c406da40 0x7fd3c415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3c8983640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c406ded0 con 0x7fd3c41432b0 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3c8983640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c40b4980 con 0x7fd3c406da40 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3c8983640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c40ce2a0 con 0x7fd3c4142de0 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3bbfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd3c41432b0 0x7fd3c406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3bbfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd3c41432b0 0x7fd3c406d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36312/0 (socket says 172.21.15.72:36312) 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3bbfff640 1 -- 172.21.15.72:0/2879639822 learned_addr learned my addr 172.21.15.72:0/2879639822 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3c259c640 1 --2- 172.21.15.72:0/2879639822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 0x7fd3c40cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.706+0000 7fd3c2d9d640 1 --2- 172.21.15.72:0/2879639822 >> v2:172.21.15.113:3300/0 conn(0x7fd3c406da40 0x7fd3c415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.707+0000 7fd3bbfff640 1 -- 172.21.15.72:0/2879639822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 msgr2=0x7fd3c40cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.707+0000 7fd3bbfff640 1 --2- 172.21.15.72:0/2879639822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 0x7fd3c40cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.707+0000 7fd3bbfff640 1 -- 172.21.15.72:0/2879639822 >> v2:172.21.15.113:3300/0 conn(0x7fd3c406da40 msgr2=0x7fd3c415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.707+0000 7fd3bbfff640 1 --2- 172.21.15.72:0/2879639822 >> v2:172.21.15.113:3300/0 conn(0x7fd3c406da40 0x7fd3c415dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.707+0000 7fd3bbfff640 1 -- 172.21.15.72:0/2879639822 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c40b4050 con 0x7fd3c41432b0 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.708+0000 7fd3bbfff640 1 --2- 172.21.15.72:0/2879639822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c41432b0 0x7fd3c406d500 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3c415ec80 tx=0x7fd3ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adf00a421cb1cbd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.708+0000 7fd3c1d9b640 1 -- 172.21.15.72:0/2879639822 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0317e0 con 0x7fd3c41432b0 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.708+0000 7fd3c1d9b640 1 -- 172.21.15.72:0/2879639822 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac031940 con 0x7fd3c41432b0 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.708+0000 7fd3c1d9b640 1 -- 172.21.15.72:0/2879639822 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac031cd0 con 0x7fd3c41432b0 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 -- 172.21.15.72:0/2879639822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c41432b0 msgr2=0x7fd3c406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 --2- 172.21.15.72:0/2879639822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c41432b0 0x7fd3c406d500 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3c415ec80 tx=0x7fd3ac02b880 comp rx=0 tx=0).stop 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 -- 172.21.15.72:0/2879639822 shutdown_connections 2023-12-12T09:53:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 --2- 172.21.15.72:0/2879639822 >> v2:172.21.15.113:3300/0 conn(0x7fd3c406da40 0x7fd3c415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 --2- 172.21.15.72:0/2879639822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c41432b0 0x7fd3c406d500 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 --2- 172.21.15.72:0/2879639822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 0x7fd3c40cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 -- 172.21.15.72:0/2879639822 >> 172.21.15.72:0/2879639822 conn(0x7fd3c40cb850 msgr2=0x7fd3c40cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:11.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 -- 172.21.15.72:0/2879639822 shutdown_connections 2023-12-12T09:53:11.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.709+0000 7fd3c8983640 1 -- 172.21.15.72:0/2879639822 wait complete. 2023-12-12T09:53:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.710+0000 7fd3c8983640 1 Processor -- start 2023-12-12T09:53:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.711+0000 7fd3c8983640 1 -- start start 2023-12-12T09:53:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.711+0000 7fd3c8983640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c406da40 0x7fd3c4141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.711+0000 7fd3c8983640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 0x7fd3c41f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.711+0000 7fd3c8983640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3c41432b0 0x7fd3c41f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.711+0000 7fd3c8983640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c41f18d0 con 0x7fd3c406da40 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.711+0000 7fd3c8983640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c413fa20 con 0x7fd3c41432b0 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.711+0000 7fd3c8983640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c413fd00 con 0x7fd3c4142de0 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c406da40 0x7fd3c4141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c406da40 0x7fd3c4141170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36324/0 (socket says 172.21.15.72:36324) 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 -- 172.21.15.72:0/946383742 learned_addr learned my addr 172.21.15.72:0/946383742 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 -- 172.21.15.72:0/946383742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 msgr2=0x7fd3c41f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 0x7fd3c41f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 -- 172.21.15.72:0/946383742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3c41432b0 msgr2=0x7fd3c41f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3c41432b0 0x7fd3c41f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 -- 172.21.15.72:0/946383742 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac0094d0 con 0x7fd3c406da40 2023-12-12T09:53:11.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.712+0000 7fd3c259c640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c406da40 0x7fd3c4141170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400d800 tx=0x7fd3b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:11.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.713+0000 7fd3baffd640 1 -- 172.21.15.72:0/946383742 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400b6c0 con 0x7fd3c406da40 2023-12-12T09:53:11.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.713+0000 7fd3baffd640 1 -- 172.21.15.72:0/946383742 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b400b820 con 0x7fd3c406da40 2023-12-12T09:53:11.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.713+0000 7fd3baffd640 1 -- 172.21.15.72:0/946383742 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400bbb0 con 0x7fd3c406da40 2023-12-12T09:53:11.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.713+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c413ffe0 con 0x7fd3c406da40 2023-12-12T09:53:11.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.713+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c413f710 con 0x7fd3c406da40 2023-12-12T09:53:11.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.714+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c40745d0 con 0x7fd3c406da40 2023-12-12T09:53:11.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.715+0000 7fd3baffd640 1 -- 172.21.15.72:0/946383742 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3b400bd10 con 0x7fd3c406da40 2023-12-12T09:53:11.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.715+0000 7fd3baffd640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3a406ae40 0x7fd3a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:11.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.715+0000 7fd3baffd640 1 -- 172.21.15.72:0/946383742 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd3b4089fe0 con 0x7fd3c406da40 2023-12-12T09:53:11.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.716+0000 7fd3bbfff640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3a406ae40 0x7fd3a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:11.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.716+0000 7fd3bbfff640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3a406ae40 0x7fd3a406d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac036000 tx=0x7fd3ac030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:11.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.720+0000 7fd3baffd640 1 -- 172.21.15.72:0/946383742 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3b4054c30 con 0x7fd3c406da40 2023-12-12T09:53:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.910+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3c40cc650 con 0x7fd3c406da40 2023-12-12T09:53:11.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.911+0000 7fd3baffd640 1 -- 172.21.15.72:0/946383742 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3b4058250 con 0x7fd3c406da40 2023-12-12T09:53:11.913 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:11.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3a406ae40 msgr2=0x7fd3a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3a406ae40 0x7fd3a406d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac036000 tx=0x7fd3ac030000 comp rx=0 tx=0).stop 2023-12-12T09:53:11.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c406da40 msgr2=0x7fd3c4141170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:11.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c406da40 0x7fd3c4141170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400d800 tx=0x7fd3b400dcc0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 shutdown_connections 2023-12-12T09:53:11.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3a406ae40 0x7fd3a406d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3c41432b0 0x7fd3c41f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3c4142de0 0x7fd3c41f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 --2- 172.21.15.72:0/946383742 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3c406da40 0x7fd3c4141170 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:11.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 >> 172.21.15.72:0/946383742 conn(0x7fd3c40cb850 msgr2=0x7fd3c40af890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:11.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 shutdown_connections 2023-12-12T09:53:11.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:11.914+0000 7fd3c8983640 1 -- 172.21.15.72:0/946383742 wait complete. 2023-12-12T09:53:11.934 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:12.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.147+0000 7fa6cab16640 1 Processor -- start 2023-12-12T09:53:12.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.147+0000 7fa6cab16640 1 -- start start 2023-12-12T09:53:12.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.148+0000 7fa6cab16640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa6c4070450 0x7fa6c40ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.148+0000 7fa6cab16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c4070830 0x7fa6c40cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.148+0000 7fa6cab16640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c40ae5b0 con 0x7fa6c40ce030 2023-12-12T09:53:12.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.148+0000 7fa6cab16640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c40cc860 con 0x7fa6c4070450 2023-12-12T09:53:12.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.148+0000 7fa6cab16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40cca70 con 0x7fa6c4070830 2023-12-12T09:53:12.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c88b2640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa6c4070450 0x7fa6c40ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c90b3640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fa6c40ce030 0x7fa6c40c9a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40026/0 (socket says 172.21.15.72:40026) 2023-12-12T09:53:12.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c90b3640 1 -- 172.21.15.72:0/4268693642 learned_addr learned my addr 172.21.15.72:0/4268693642 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:12.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c3fff640 1 --2- 172.21.15.72:0/4268693642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c4070830 0x7fa6c40cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c88b2640 1 -- 172.21.15.72:0/4268693642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c4070830 msgr2=0x7fa6c40cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:12.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c88b2640 1 --2- 172.21.15.72:0/4268693642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c4070830 0x7fa6c40cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c88b2640 1 -- 172.21.15.72:0/4268693642 >> v1:172.21.15.72:6789/0 conn(0x7fa6c40ce030 legacy=0x7fa6c40c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c88b2640 1 -- 172.21.15.72:0/4268693642 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40ccd50 con 0x7fa6c4070450 2023-12-12T09:53:12.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.149+0000 7fa6c3fff640 1 --2- 172.21.15.72:0/4268693642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c4070830 0x7fa6c40cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.151+0000 7fa6c88b2640 1 --2- 172.21.15.72:0/4268693642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40ae070 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800a940 tx=0x7fa6b802ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98312751257efe3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.151+0000 7fa6c37fe640 1 -- 172.21.15.72:0/4268693642 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b802dbc0 con 0x7fa6c4070450 2023-12-12T09:53:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.151+0000 7fa6c37fe640 1 -- 172.21.15.72:0/4268693642 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b802dd20 con 0x7fa6c4070450 2023-12-12T09:53:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.151+0000 7fa6c37fe640 1 -- 172.21.15.72:0/4268693642 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8035660 con 0x7fa6c4070450 2023-12-12T09:53:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.151+0000 7fa6cab16640 1 -- 172.21.15.72:0/4268693642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 msgr2=0x7fa6c40ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.151+0000 7fa6cab16640 1 --2- 172.21.15.72:0/4268693642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40ae070 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800a940 tx=0x7fa6b802ce10 comp rx=0 tx=0).stop 2023-12-12T09:53:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.152+0000 7fa6cab16640 1 -- 172.21.15.72:0/4268693642 shutdown_connections 2023-12-12T09:53:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.152+0000 7fa6cab16640 1 --2- 172.21.15.72:0/4268693642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c4070830 0x7fa6c40cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.152+0000 7fa6cab16640 1 --2- 172.21.15.72:0/4268693642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40ae070 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.152+0000 7fa6cab16640 1 -- 172.21.15.72:0/4268693642 >> 172.21.15.72:0/4268693642 conn(0x7fa6c40cee70 msgr2=0x7fa6c40cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.152+0000 7fa6cab16640 1 -- 172.21.15.72:0/4268693642 shutdown_connections 2023-12-12T09:53:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.152+0000 7fa6cab16640 1 -- 172.21.15.72:0/4268693642 wait complete. 2023-12-12T09:53:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.154+0000 7fa6cab16640 1 Processor -- start 2023-12-12T09:53:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.154+0000 7fa6cab16640 1 -- start start 2023-12-12T09:53:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6cab16640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6cab16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c4070830 0x7fa6c41e39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6cab16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ce030 0x7fa6c4156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6cab16640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40b2ce0 con 0x7fa6c4070450 2023-12-12T09:53:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6cab16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40b1e70 con 0x7fa6c4070830 2023-12-12T09:53:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6cab16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41e3ef0 con 0x7fa6c40ce030 2023-12-12T09:53:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6c88b2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6c88b2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36346/0 (socket says 172.21.15.72:36346) 2023-12-12T09:53:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6c88b2640 1 -- 172.21.15.72:0/1600075877 learned_addr learned my addr 172.21.15.72:0/1600075877 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6c88b2640 1 -- 172.21.15.72:0/1600075877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ce030 msgr2=0x7fa6c4156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6c88b2640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ce030 0x7fa6c4156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6c88b2640 1 -- 172.21.15.72:0/1600075877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c4070830 msgr2=0x7fa6c41e39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6c88b2640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c4070830 0x7fa6c41e39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.155+0000 7fa6c88b2640 1 -- 172.21.15.72:0/1600075877 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b800a5a0 con 0x7fa6c4070450 2023-12-12T09:53:12.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.156+0000 7fa6c88b2640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40b1930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8038040 tx=0x7fa6b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:12.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.156+0000 7fa6c17fa640 1 -- 172.21.15.72:0/1600075877 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b803e610 con 0x7fa6c4070450 2023-12-12T09:53:12.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.156+0000 7fa6c17fa640 1 -- 172.21.15.72:0/1600075877 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b802de90 con 0x7fa6c4070450 2023-12-12T09:53:12.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.156+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40b2060 con 0x7fa6c4070450 2023-12-12T09:53:12.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.156+0000 7fa6c17fa640 1 -- 172.21.15.72:0/1600075877 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b803e9b0 con 0x7fa6c4070450 2023-12-12T09:53:12.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.156+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c40b2620 con 0x7fa6c4070450 2023-12-12T09:53:12.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.157+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c406de90 con 0x7fa6c4070450 2023-12-12T09:53:12.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.158+0000 7fa6c17fa640 1 -- 172.21.15.72:0/1600075877 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6b803c040 con 0x7fa6c4070450 2023-12-12T09:53:12.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.159+0000 7fa6c17fa640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa6a806ae40 0x7fa6a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.159+0000 7fa6c17fa640 1 -- 172.21.15.72:0/1600075877 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa6b80abae0 con 0x7fa6c4070450 2023-12-12T09:53:12.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.160+0000 7fa6c3fff640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa6a806ae40 0x7fa6a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.160+0000 7fa6c3fff640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa6a806ae40 0x7fa6a806d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0006fd0 tx=0x7fa6b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:12.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.164+0000 7fa6c17fa640 1 -- 172.21.15.72:0/1600075877 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6b8076880 con 0x7fa6c4070450 2023-12-12T09:53:12.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.353+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6c4066eb0 con 0x7fa6c4070450 2023-12-12T09:53:12.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.354+0000 7fa6c17fa640 1 -- 172.21.15.72:0/1600075877 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6b8079ea0 con 0x7fa6c4070450 2023-12-12T09:53:12.355 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:12.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.356+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa6a806ae40 msgr2=0x7fa6a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.356+0000 7fa6cab16640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa6a806ae40 0x7fa6a806d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0006fd0 tx=0x7fa6b0008040 comp rx=0 tx=0).stop 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.356+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 msgr2=0x7fa6c40b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.356+0000 7fa6cab16640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40b1930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8038040 tx=0x7fa6b80046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.356+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 shutdown_connections 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.357+0000 7fa6cab16640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa6a806ae40 0x7fa6a806d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.357+0000 7fa6cab16640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ce030 0x7fa6c4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.357+0000 7fa6cab16640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c4070830 0x7fa6c41e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.357+0000 7fa6cab16640 1 --2- 172.21.15.72:0/1600075877 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c4070450 0x7fa6c40b1930 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.357+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 >> 172.21.15.72:0/1600075877 conn(0x7fa6c40cee70 msgr2=0x7fa6c406c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.357+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 shutdown_connections 2023-12-12T09:53:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.357+0000 7fa6cab16640 1 -- 172.21.15.72:0/1600075877 wait complete. 2023-12-12T09:53:12.377 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:12.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.594+0000 7f9e8f116640 1 Processor -- start 2023-12-12T09:53:12.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.594+0000 7f9e8f116640 1 -- start start 2023-12-12T09:53:12.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.596+0000 7f9e8f116640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9e88143810 0x7f9e880ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.596+0000 7f9e8f116640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e88143bf0 con 0x7f9e8806dc10 2023-12-12T09:53:12.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.596+0000 7f9e8f116640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e880ae520 con 0x7f9e880cd7d0 2023-12-12T09:53:12.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.596+0000 7f9e8f116640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e880ae800 con 0x7f9e88143810 2023-12-12T09:53:12.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.596+0000 7f9e8ceb2640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9e88143810 0x7f9e880ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.596+0000 7f9e8d6b3640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f9e8806dc10 0x7f9e88160f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40030/0 (socket says 172.21.15.72:40030) 2023-12-12T09:53:12.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.596+0000 7f9e8d6b3640 1 -- 172.21.15.72:0/1351195075 learned_addr learned my addr 172.21.15.72:0/1351195075 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:12.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.597+0000 7f9e8ceb2640 1 -- 172.21.15.72:0/1351195075 >> v1:172.21.15.113:6789/0 conn(0x7f9e880cd7d0 legacy=0x7f9e8806d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:12.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.597+0000 7f9e8ceb2640 1 -- 172.21.15.72:0/1351195075 >> v1:172.21.15.72:6789/0 conn(0x7f9e8806dc10 legacy=0x7f9e88160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.597+0000 7f9e8ceb2640 1 -- 172.21.15.72:0/1351195075 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880cccf0 con 0x7f9e88143810 2023-12-12T09:53:12.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.597+0000 7f9e877fe640 1 -- 172.21.15.72:0/1351195075 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626408425 0 0) 0x7f9e88143bf0 con 0x7f9e8806dc10 2023-12-12T09:53:12.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.598+0000 7f9e8ceb2640 1 --2- 172.21.15.72:0/1351195075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e880ada80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0097a0 tx=0x7f9e7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c088f02e3a804d47 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:12.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.598+0000 7f9e877fe640 1 -- 172.21.15.72:0/1351195075 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c031830 con 0x7f9e88143810 2023-12-12T09:53:12.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.598+0000 7f9e877fe640 1 -- 172.21.15.72:0/1351195075 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c031990 con 0x7f9e88143810 2023-12-12T09:53:12.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.598+0000 7f9e877fe640 1 -- 172.21.15.72:0/1351195075 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c031d20 con 0x7f9e88143810 2023-12-12T09:53:12.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.599+0000 7f9e8f116640 1 -- 172.21.15.72:0/1351195075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 msgr2=0x7f9e880ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.599+0000 7f9e8f116640 1 --2- 172.21.15.72:0/1351195075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e880ada80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0097a0 tx=0x7f9e7c02b8d0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.599+0000 7f9e8f116640 1 -- 172.21.15.72:0/1351195075 shutdown_connections 2023-12-12T09:53:12.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.599+0000 7f9e8f116640 1 --2- 172.21.15.72:0/1351195075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e880ada80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.599+0000 7f9e8f116640 1 -- 172.21.15.72:0/1351195075 >> 172.21.15.72:0/1351195075 conn(0x7f9e880cb850 msgr2=0x7f9e880cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:12.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.599+0000 7f9e8f116640 1 -- 172.21.15.72:0/1351195075 shutdown_connections 2023-12-12T09:53:12.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.599+0000 7f9e8f116640 1 -- 172.21.15.72:0/1351195075 wait complete. 2023-12-12T09:53:12.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.601+0000 7f9e8f116640 1 Processor -- start 2023-12-12T09:53:12.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.601+0000 7f9e8f116640 1 -- start start 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8f116640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8806dc10 0x7f9e881e3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8f116640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e880cd7d0 0x7f9e88140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8f116640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e8813f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8f116640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8813da00 con 0x7f9e88143810 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8f116640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8813fe80 con 0x7f9e8806dc10 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8f116640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88140160 con 0x7f9e880cd7d0 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8d6b3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e8813f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8d6b3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e8813f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36374/0 (socket says 172.21.15.72:36374) 2023-12-12T09:53:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8d6b3640 1 -- 172.21.15.72:0/891726651 learned_addr learned my addr 172.21.15.72:0/891726651 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:12.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e8ceb2640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8806dc10 0x7f9e881e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.602+0000 7f9e87fff640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e880cd7d0 0x7f9e88140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8d6b3640 1 -- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e880cd7d0 msgr2=0x7f9e88140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8d6b3640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e880cd7d0 0x7f9e88140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8d6b3640 1 -- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8806dc10 msgr2=0x7f9e881e3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8d6b3640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8806dc10 0x7f9e881e3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8d6b3640 1 -- 172.21.15.72:0/891726651 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c009400 con 0x7f9e88143810 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8ceb2640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8806dc10 0x7f9e881e3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e87fff640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e880cd7d0 0x7f9e88140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8d6b3640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e8813f780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c7e0 tx=0x7f9e8000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e857fa640 1 -- 172.21.15.72:0/891726651 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000a640 con 0x7f9e88143810 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e857fa640 1 -- 172.21.15.72:0/891726651 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8000a7c0 con 0x7f9e88143810 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e857fa640 1 -- 172.21.15.72:0/891726651 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000ab70 con 0x7f9e88143810 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e88140440 con 0x7f9e88143810 2023-12-12T09:53:12.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.603+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e88140a00 con 0x7f9e88143810 2023-12-12T09:53:12.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.605+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e880745d0 con 0x7f9e88143810 2023-12-12T09:53:12.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.605+0000 7f9e857fa640 1 -- 172.21.15.72:0/891726651 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e8000acd0 con 0x7f9e88143810 2023-12-12T09:53:12.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.607+0000 7f9e857fa640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9e6c06ae40 0x7f9e6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:12.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.607+0000 7f9e857fa640 1 -- 172.21.15.72:0/891726651 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e80088e70 con 0x7f9e88143810 2023-12-12T09:53:12.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.607+0000 7f9e8ceb2640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9e6c06ae40 0x7f9e6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:12.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.608+0000 7f9e8ceb2640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9e6c06ae40 0x7f9e6c06d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c036000 tx=0x7f9e7c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:12.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.611+0000 7f9e857fa640 1 -- 172.21.15.72:0/891726651 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e80053b50 con 0x7f9e88143810 2023-12-12T09:53:12.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.800+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9e88140c20 con 0x7f9e88143810 2023-12-12T09:53:12.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.801+0000 7f9e857fa640 1 -- 172.21.15.72:0/891726651 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9e80018020 con 0x7f9e88143810 2023-12-12T09:53:12.803 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9e6c06ae40 msgr2=0x7f9e6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9e6c06ae40 0x7f9e6c06d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c036000 tx=0x7f9e7c03c040 comp rx=0 tx=0).stop 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 msgr2=0x7f9e8813f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e8813f780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c7e0 tx=0x7f9e8000cca0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 shutdown_connections 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9e6c06ae40 0x7f9e6c06d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e88143810 0x7f9e8813f780 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e880cd7d0 0x7f9e88140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 --2- 172.21.15.72:0/891726651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8806dc10 0x7f9e881e3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 >> 172.21.15.72:0/891726651 conn(0x7f9e880cb850 msgr2=0x7f9e880afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:12.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 shutdown_connections 2023-12-12T09:53:12.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:12.804+0000 7f9e8f116640 1 -- 172.21.15.72:0/891726651 wait complete. 2023-12-12T09:53:12.856 INFO:tasks.cephfs_test_runner:Starting test: test_dir_deletion (tasks.cephfs.test_strays.TestStrays) That when deleting a bunch of dentries and the containing 2023-12-12T09:53:12.856 INFO:tasks.cephfs_test_runner:test_dir_deletion (tasks.cephfs.test_strays.TestStrays) 2023-12-12T09:53:12.857 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:13.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.076+0000 7f05a6a6b640 1 Processor -- start 2023-12-12T09:53:13.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.076+0000 7f05a6a6b640 1 -- start start 2023-12-12T09:53:13.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.078+0000 7f05a6a6b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05a0143e00 0x7f05a00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.078+0000 7f05a6a6b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05a0072180 0x7f05a00b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.078+0000 7f05a6a6b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a00b3bb0 con 0x7f05a006c410 2023-12-12T09:53:13.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.078+0000 7f05a6a6b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a01441e0 con 0x7f05a0143e00 2023-12-12T09:53:13.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.078+0000 7f05a6a6b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0072680 con 0x7f05a0072180 2023-12-12T09:53:13.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.078+0000 7f05a5008640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f05a006c410 0x7f05a0070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40036/0 (socket says 172.21.15.72:40036) 2023-12-12T09:53:13.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.078+0000 7f05a5008640 1 -- 172.21.15.72:0/1645080992 learned_addr learned my addr 172.21.15.72:0/1645080992 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:13.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f05a4807640 1 --2- 172.21.15.72:0/1645080992 >> v2:172.21.15.113:3300/0 conn(0x7f05a0143e00 0x7f05a00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059ffff640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05a0072180 0x7f05a00b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713637709 0 0) 0x7f05a00b3bb0 con 0x7f05a006c410 2023-12-12T09:53:13.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0590003520 con 0x7f05a006c410 2023-12-12T09:53:13.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2484056605 0 0) 0x7f0590003520 con 0x7f05a006c410 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05a0072180 msgr2=0x7f05a00b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05a0072180 0x7f05a00b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 >> v2:172.21.15.113:3300/0 conn(0x7f05a0143e00 msgr2=0x7f05a00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 --2- 172.21.15.72:0/1645080992 >> v2:172.21.15.113:3300/0 conn(0x7f05a0143e00 0x7f05a00b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a00b1e30 con 0x7f05a006c410 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f05980019f0 con 0x7f05a006c410 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f05a4807640 1 --2- 172.21.15.72:0/1645080992 >> v2:172.21.15.113:3300/0 conn(0x7f05a0143e00 0x7f05a00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 >> v1:172.21.15.72:6789/0 conn(0x7f05a006c410 legacy=0x7f05a0070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0590003970 0x7f0590003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05a0072180 0x7f05900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0590004d00 0x7f0590005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a01441e0 con 0x7f0590004d00 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0072680 con 0x7f0590003970 2023-12-12T09:53:13.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.079+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059000ba00 con 0x7f05a0072180 2023-12-12T09:53:13.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.080+0000 7f05a5008640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0590004d00 0x7f0590005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.080+0000 7f059ffff640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05a0072180 0x7f05900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.080+0000 7f05a4807640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0590003970 0x7f0590003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.080+0000 7f05a5008640 1 -- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05a0072180 msgr2=0x7f05900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.080+0000 7f05a5008640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05a0072180 0x7f05900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.080+0000 7f05a5008640 1 -- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0590003970 msgr2=0x7f0590003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.080+0000 7f05a5008640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0590003970 0x7f0590003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.080+0000 7f05a5008640 1 -- 172.21.15.72:0/1645080992 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590003520 con 0x7f0590004d00 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.081+0000 7f05a5008640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0590004d00 0x7f0590005180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f059800a780 tx=0x7f05900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6242b1e4885dda server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.081+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598034630 con 0x7f0590004d00 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059802f070 con 0x7f0590004d00 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f059f7fe640 1 -- 172.21.15.72:0/1645080992 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980349d0 con 0x7f0590004d00 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1645080992 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0590004d00 msgr2=0x7f0590005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f05a6a6b640 1 --2- 172.21.15.72:0/1645080992 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0590004d00 0x7f0590005180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f059800a780 tx=0x7f05900057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f05a5008640 1 -- 172.21.15.72:0/1645080992 reap_dead start 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1645080992 shutdown_connections 2023-12-12T09:53:13.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1645080992 >> 172.21.15.72:0/1645080992 conn(0x7f05a00b0a90 msgr2=0x7f05a00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:13.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1645080992 shutdown_connections 2023-12-12T09:53:13.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.082+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1645080992 wait complete. 2023-12-12T09:53:13.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.084+0000 7f05a6a6b640 1 Processor -- start 2023-12-12T09:53:13.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.084+0000 7f05a6a6b640 1 -- start start 2023-12-12T09:53:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f05a6a6b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0590004d00 0x7f05a0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f05a6a6b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f05a013c1a0 0x7f05a0139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f05a6a6b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a013d760 0x7f05a0139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f05a6a6b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a013bd80 con 0x7f05a013c1a0 2023-12-12T09:53:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f05a6a6b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a013b440 con 0x7f05a013d760 2023-12-12T09:53:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f05a6a6b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a015aaf0 con 0x7f0590004d00 2023-12-12T09:53:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f059ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f05a013c1a0 0x7f05a0139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f059ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f05a013c1a0 0x7f05a0139680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36392/0 (socket says 172.21.15.72:36392) 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.085+0000 7f059ffff640 1 -- 172.21.15.72:0/1568240553 learned_addr learned my addr 172.21.15.72:0/1568240553 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f05a4807640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0590004d00 0x7f05a0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059ffff640 1 -- 172.21.15.72:0/1568240553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0590004d00 msgr2=0x7f05a0159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059ffff640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0590004d00 0x7f05a0159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059ffff640 1 -- 172.21.15.72:0/1568240553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a013d760 msgr2=0x7f05a0139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f05a5008640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a013d760 0x7f05a0139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059ffff640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a013d760 0x7f05a0139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059ffff640 1 -- 172.21.15.72:0/1568240553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a013dc60 con 0x7f05a013c1a0 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059ffff640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f05a013c1a0 0x7f05a0139680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f058c011600 tx=0x7f058c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059d7fa640 1 -- 172.21.15.72:0/1568240553 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00f920 con 0x7f05a013c1a0 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059d7fa640 1 -- 172.21.15.72:0/1568240553 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c00fa80 con 0x7f05a013c1a0 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f059d7fa640 1 -- 172.21.15.72:0/1568240553 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00fdf0 con 0x7f05a013c1a0 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.086+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a013b6c0 con 0x7f05a013c1a0 2023-12-12T09:53:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.087+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a015a460 con 0x7f05a013c1a0 2023-12-12T09:53:13.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.087+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a013bff0 con 0x7f05a013c1a0 2023-12-12T09:53:13.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.088+0000 7f059d7fa640 1 -- 172.21.15.72:0/1568240553 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f058c015460 con 0x7f05a013c1a0 2023-12-12T09:53:13.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.089+0000 7f059d7fa640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f058406ae40 0x7f058406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.089+0000 7f059d7fa640 1 -- 172.21.15.72:0/1568240553 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f058c089ec0 con 0x7f05a013c1a0 2023-12-12T09:53:13.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.090+0000 7f05a4807640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f058406ae40 0x7f058406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.090+0000 7f05a4807640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f058406ae40 0x7f058406d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0594006fd0 tx=0x7f0594008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.094+0000 7f059d7fa640 1 -- 172.21.15.72:0/1568240553 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f058c054ba0 con 0x7f05a013c1a0 2023-12-12T09:53:13.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.287+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05a00745d0 con 0x7f05a013c1a0 2023-12-12T09:53:13.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.288+0000 7f059d7fa640 1 -- 172.21.15.72:0/1568240553 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7f058c022090 con 0x7f05a013c1a0 2023-12-12T09:53:13.291 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:13.291 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:13.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f058406ae40 msgr2=0x7f058406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f058406ae40 0x7f058406d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0594006fd0 tx=0x7f0594008040 comp rx=0 tx=0).stop 2023-12-12T09:53:13.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f05a013c1a0 msgr2=0x7f05a0139680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f05a013c1a0 0x7f05a0139680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f058c011600 tx=0x7f058c011ac0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 shutdown_connections 2023-12-12T09:53:13.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f058406ae40 0x7f058406d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a013d760 0x7f05a0139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f05a013c1a0 0x7f05a0139680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 --2- 172.21.15.72:0/1568240553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0590004d00 0x7f05a0159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 >> 172.21.15.72:0/1568240553 conn(0x7f05a00b0a90 msgr2=0x7f05a00ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:13.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 shutdown_connections 2023-12-12T09:53:13.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.291+0000 7f05a6a6b640 1 -- 172.21.15.72:0/1568240553 wait complete. 2023-12-12T09:53:13.311 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:13.312 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:13.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.527+0000 7f4a39d33640 1 Processor -- start 2023-12-12T09:53:13.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.527+0000 7f4a39d33640 1 -- start start 2023-12-12T09:53:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.529+0000 7f4a39d33640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4a3415af50 0x7f4a3406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.529+0000 7f4a39d33640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a340ce7e0 0x7f4a3415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.529+0000 7f4a39d33640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a34071cd0 con 0x7f4a3406a460 2023-12-12T09:53:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.529+0000 7f4a39d33640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a340717e0 con 0x7f4a3415af50 2023-12-12T09:53:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.529+0000 7f4a39d33640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3406c6b0 con 0x7f4a340ce7e0 2023-12-12T09:53:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.529+0000 7f4a337fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4a3415af50 0x7f4a3406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.529+0000 7f4a337fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4a3415af50 0x7f4a3406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36394/0 (socket says 172.21.15.72:36394) 2023-12-12T09:53:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.529+0000 7f4a337fe640 1 -- 172.21.15.72:0/2570068783 learned_addr learned my addr 172.21.15.72:0/2570068783 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:13.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.530+0000 7f4a337fe640 1 -- 172.21.15.72:0/2570068783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a340ce7e0 msgr2=0x7f4a3415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.530+0000 7f4a2bfff640 1 --2- 172.21.15.72:0/2570068783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a340ce7e0 0x7f4a3415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.530+0000 7f4a337fe640 1 --2- 172.21.15.72:0/2570068783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a340ce7e0 0x7f4a3415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.530+0000 7f4a337fe640 1 -- 172.21.15.72:0/2570068783 >> v1:172.21.15.113:6789/0 conn(0x7f4a3406a460 legacy=0x7f4a340ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.530+0000 7f4a337fe640 1 -- 172.21.15.72:0/2570068783 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a340719f0 con 0x7f4a3415af50 2023-12-12T09:53:13.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.530+0000 7f4a2bfff640 1 --2- 172.21.15.72:0/2570068783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a340ce7e0 0x7f4a3415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.531+0000 7f4a337fe640 1 --2- 172.21.15.72:0/2570068783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3415af50 0x7f4a3406c170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a24009870 tx=0x7f4a2402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4a0822259320d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.531+0000 7f4a32ffd640 1 -- 172.21.15.72:0/2570068783 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24031740 con 0x7f4a3415af50 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.531+0000 7f4a32ffd640 1 -- 172.21.15.72:0/2570068783 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a240318a0 con 0x7f4a3415af50 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.531+0000 7f4a32ffd640 1 -- 172.21.15.72:0/2570068783 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24031c30 con 0x7f4a3415af50 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.532+0000 7f4a39d33640 1 -- 172.21.15.72:0/2570068783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3415af50 msgr2=0x7f4a3406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.532+0000 7f4a39d33640 1 --2- 172.21.15.72:0/2570068783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3415af50 0x7f4a3406c170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a24009870 tx=0x7f4a2402b7e0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.532+0000 7f4a39d33640 1 -- 172.21.15.72:0/2570068783 shutdown_connections 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.532+0000 7f4a39d33640 1 --2- 172.21.15.72:0/2570068783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a340ce7e0 0x7f4a3415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.532+0000 7f4a39d33640 1 --2- 172.21.15.72:0/2570068783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3415af50 0x7f4a3406c170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.532+0000 7f4a39d33640 1 -- 172.21.15.72:0/2570068783 >> 172.21.15.72:0/2570068783 conn(0x7f4a340cee70 msgr2=0x7f4a340cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.532+0000 7f4a39d33640 1 -- 172.21.15.72:0/2570068783 shutdown_connections 2023-12-12T09:53:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.532+0000 7f4a39d33640 1 -- 172.21.15.72:0/2570068783 wait complete. 2023-12-12T09:53:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.534+0000 7f4a39d33640 1 Processor -- start 2023-12-12T09:53:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.534+0000 7f4a39d33640 1 -- start start 2023-12-12T09:53:13.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a39d33640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3406a460 0x7f4a3415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a39d33640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a340ce7e0 0x7f4a340c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a39d33640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3415af50 0x7f4a341e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a39d33640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340c30d0 con 0x7f4a3406a460 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a39d33640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340c33b0 con 0x7f4a340ce7e0 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a39d33640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340c3690 con 0x7f4a3415af50 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a337fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3406a460 0x7f4a3415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a337fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3406a460 0x7f4a3415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36398/0 (socket says 172.21.15.72:36398) 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.535+0000 7f4a337fe640 1 -- 172.21.15.72:0/1608025363 learned_addr learned my addr 172.21.15.72:0/1608025363 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a2bfff640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a340ce7e0 0x7f4a340c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a337fe640 1 -- 172.21.15.72:0/1608025363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3415af50 msgr2=0x7f4a341e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a337fe640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3415af50 0x7f4a341e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a337fe640 1 -- 172.21.15.72:0/1608025363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a340ce7e0 msgr2=0x7f4a340c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a337fe640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a340ce7e0 0x7f4a340c2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a337fe640 1 -- 172.21.15.72:0/1608025363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240094d0 con 0x7f4a3406a460 2023-12-12T09:53:13.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a337fe640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3406a460 0x7f4a3415a0e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a24000c00 tx=0x7f4a240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a30ff9640 1 -- 172.21.15.72:0/1608025363 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24033d60 con 0x7f4a3406a460 2023-12-12T09:53:13.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a30ff9640 1 -- 172.21.15.72:0/1608025363 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2403c400 con 0x7f4a3406a460 2023-12-12T09:53:13.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a340b2900 con 0x7f4a3406a460 2023-12-12T09:53:13.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a30ff9640 1 -- 172.21.15.72:0/1608025363 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2403c7b0 con 0x7f4a3406a460 2023-12-12T09:53:13.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.536+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a340cabc0 con 0x7f4a3406a460 2023-12-12T09:53:13.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.537+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a340b2b40 con 0x7f4a3406a460 2023-12-12T09:53:13.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.538+0000 7f4a30ff9640 1 -- 172.21.15.72:0/1608025363 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a2403a040 con 0x7f4a3406a460 2023-12-12T09:53:13.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.540+0000 7f4a30ff9640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4a1406ae40 0x7f4a1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.540+0000 7f4a30ff9640 1 -- 172.21.15.72:0/1608025363 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4a240ab6d0 con 0x7f4a3406a460 2023-12-12T09:53:13.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.540+0000 7f4a2bfff640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4a1406ae40 0x7f4a1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.541+0000 7f4a2bfff640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4a1406ae40 0x7f4a1406d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a3406e330 tx=0x7f4a1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.545+0000 7f4a30ff9640 1 -- 172.21.15.72:0/1608025363 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a340b2b40 con 0x7f4a3406a460 2023-12-12T09:53:13.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.737+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a34074450 con 0x7f4a3406a460 2023-12-12T09:53:13.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.738+0000 7f4a30ff9640 1 -- 172.21.15.72:0/1608025363 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a24076440 con 0x7f4a3406a460 2023-12-12T09:53:13.740 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:13.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4a1406ae40 msgr2=0x7f4a1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4a1406ae40 0x7f4a1406d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a3406e330 tx=0x7f4a1c008040 comp rx=0 tx=0).stop 2023-12-12T09:53:13.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3406a460 msgr2=0x7f4a3415a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3406a460 0x7f4a3415a0e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a24000c00 tx=0x7f4a240046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 shutdown_connections 2023-12-12T09:53:13.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4a1406ae40 0x7f4a1406d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3415af50 0x7f4a341e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a340ce7e0 0x7f4a340c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 --2- 172.21.15.72:0/1608025363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a3406a460 0x7f4a3415a0e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 >> 172.21.15.72:0/1608025363 conn(0x7f4a340cee70 msgr2=0x7f4a340700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:13.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.740+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 shutdown_connections 2023-12-12T09:53:13.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.741+0000 7f4a39d33640 1 -- 172.21.15.72:0/1608025363 wait complete. 2023-12-12T09:53:13.761 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:13.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.978+0000 7f6e3b0f3640 1 Processor -- start 2023-12-12T09:53:13.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.978+0000 7f6e3b0f3640 1 -- start start 2023-12-12T09:53:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.980+0000 7f6e3b0f3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3406d770 0x7f6e340b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.980+0000 7f6e3b0f3640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3415b780 con 0x7f6e3406db50 2023-12-12T09:53:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.980+0000 7f6e3b0f3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3415b9d0 con 0x7f6e34141960 2023-12-12T09:53:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.980+0000 7f6e3b0f3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415bcb0 con 0x7f6e3406d770 2023-12-12T09:53:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.980+0000 7f6e33fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6e3406db50 0x7f6e34141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40038/0 (socket says 172.21.15.72:40038) 2023-12-12T09:53:13.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.980+0000 7f6e33fff640 1 -- 172.21.15.72:0/1533026143 learned_addr learned my addr 172.21.15.72:0/1533026143 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:13.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.980+0000 7f6e38e8f640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3406d770 0x7f6e340b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.980+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433391910 0 0) 0x7f6e3415b780 con 0x7f6e3406db50 2023-12-12T09:53:13.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e24003520 con 0x7f6e3406db50 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 441049156 0 0) 0x7f6e24003520 con 0x7f6e3406db50 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3406d770 msgr2=0x7f6e340b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3406d770 0x7f6e340b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 >> v1:172.21.15.113:6789/0 conn(0x7f6e34141960 legacy=0x7f6e340cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3415bf90 con 0x7f6e3406db50 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f6e20001670 con 0x7f6e3406db50 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 >> v1:172.21.15.72:6789/0 conn(0x7f6e3406db50 legacy=0x7f6e34141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e240039a0 0x7f6e24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3406d770 0x7f6e240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e24004d00 0x7f6e24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415bcb0 con 0x7f6e24004d00 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2400ba60 con 0x7f6e240039a0 2023-12-12T09:53:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2400bc20 con 0x7f6e3406d770 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.981+0000 7f6e39690640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e24004d00 0x7f6e24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.982+0000 7f6e38e8f640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e240039a0 0x7f6e24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.982+0000 7f6e33fff640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3406d770 0x7f6e240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.982+0000 7f6e39690640 1 -- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3406d770 msgr2=0x7f6e240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.982+0000 7f6e39690640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3406d770 0x7f6e240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.982+0000 7f6e39690640 1 -- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e240039a0 msgr2=0x7f6e24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.982+0000 7f6e39690640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e240039a0 0x7f6e24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.982+0000 7f6e39690640 1 -- 172.21.15.72:0/1533026143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e24003520 con 0x7f6e24004d00 2023-12-12T09:53:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.983+0000 7f6e39690640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e24004d00 0x7f6e24005180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c001e20 tx=0x7f6e240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de2a1dd7c587904 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.983+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c003b70 con 0x7f6e24004d00 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.983+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c003cd0 con 0x7f6e24004d00 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.983+0000 7f6e337fe640 1 -- 172.21.15.72:0/1533026143 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c005310 con 0x7f6e24004d00 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.984+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/1533026143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e24004d00 msgr2=0x7f6e24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.984+0000 7f6e3b0f3640 1 --2- 172.21.15.72:0/1533026143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e24004d00 0x7f6e24005180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c001e20 tx=0x7f6e240057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.984+0000 7f6e39690640 1 -- 172.21.15.72:0/1533026143 reap_dead start 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.984+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/1533026143 shutdown_connections 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.984+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/1533026143 >> 172.21.15.72:0/1533026143 conn(0x7f6e340cb850 msgr2=0x7f6e340cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.984+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/1533026143 shutdown_connections 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.984+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/1533026143 wait complete. 2023-12-12T09:53:13.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.986+0000 7f6e3b0f3640 1 Processor -- start 2023-12-12T09:53:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.986+0000 7f6e3b0f3640 1 -- start start 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e3b0f3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3413c3e0 0x7f6e340c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e3b0f3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e341f46f0 0x7f6e340c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e3b0f3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e34159420 0x7f6e341598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e3b0f3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34159e10 con 0x7f6e3413c3e0 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e3b0f3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340c3d70 con 0x7f6e341f46f0 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e3b0f3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e341f4bd0 con 0x7f6e34159420 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e38e8f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3413c3e0 0x7f6e340c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e38e8f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3413c3e0 0x7f6e340c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36428/0 (socket says 172.21.15.72:36428) 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e38e8f640 1 -- 172.21.15.72:0/962304860 learned_addr learned my addr 172.21.15.72:0/962304860 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:13.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e33fff640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e341f46f0 0x7f6e340c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e38e8f640 1 -- 172.21.15.72:0/962304860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e34159420 msgr2=0x7f6e341598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e38e8f640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e34159420 0x7f6e341598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e38e8f640 1 -- 172.21.15.72:0/962304860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e341f46f0 msgr2=0x7f6e340c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e38e8f640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e341f46f0 0x7f6e340c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.987+0000 7f6e38e8f640 1 -- 172.21.15.72:0/962304860 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340b3940 con 0x7f6e3413c3e0 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.988+0000 7f6e38e8f640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3413c3e0 0x7f6e340c5470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6e28011600 tx=0x7f6e28011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.988+0000 7f6e317fa640 1 -- 172.21.15.72:0/962304860 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28015870 con 0x7f6e3413c3e0 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.988+0000 7f6e317fa640 1 -- 172.21.15.72:0/962304860 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280159d0 con 0x7f6e3413c3e0 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.988+0000 7f6e317fa640 1 -- 172.21.15.72:0/962304860 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28015d40 con 0x7f6e3413c3e0 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.988+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e341f4eb0 con 0x7f6e3413c3e0 2023-12-12T09:53:13.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.988+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e341f5470 con 0x7f6e3413c3e0 2023-12-12T09:53:13.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.989+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e34067910 con 0x7f6e3413c3e0 2023-12-12T09:53:13.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.990+0000 7f6e317fa640 1 -- 172.21.15.72:0/962304860 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e2800f490 con 0x7f6e3413c3e0 2023-12-12T09:53:13.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.991+0000 7f6e317fa640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e1806ae40 0x7f6e1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:13.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.992+0000 7f6e317fa640 1 -- 172.21.15.72:0/962304860 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6e28089f20 con 0x7f6e3413c3e0 2023-12-12T09:53:13.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.992+0000 7f6e33fff640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e1806ae40 0x7f6e1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:13.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.992+0000 7f6e33fff640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e1806ae40 0x7f6e1806d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e20001eb0 tx=0x7f6e20000ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:13.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:13.996+0000 7f6e317fa640 1 -- 172.21.15.72:0/962304860 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e28054b70 con 0x7f6e3413c3e0 2023-12-12T09:53:14.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.176+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e340745d0 con 0x7f6e3413c3e0 2023-12-12T09:53:14.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.177+0000 7f6e317fa640 1 -- 172.21.15.72:0/962304860 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e28022020 con 0x7f6e3413c3e0 2023-12-12T09:53:14.179 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.179+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e1806ae40 msgr2=0x7f6e1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.179+0000 7f6e3b0f3640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e1806ae40 0x7f6e1806d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e20001eb0 tx=0x7f6e20000ef0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.179+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3413c3e0 msgr2=0x7f6e340c5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.179+0000 7f6e3b0f3640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3413c3e0 0x7f6e340c5470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6e28011600 tx=0x7f6e28011ac0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.180+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 shutdown_connections 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.180+0000 7f6e3b0f3640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e1806ae40 0x7f6e1806d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.180+0000 7f6e3b0f3640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e34159420 0x7f6e341598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.180+0000 7f6e3b0f3640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e341f46f0 0x7f6e340c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.180+0000 7f6e3b0f3640 1 --2- 172.21.15.72:0/962304860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3413c3e0 0x7f6e340c5470 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.180+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 >> 172.21.15.72:0/962304860 conn(0x7f6e340cb850 msgr2=0x7f6e3413f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:14.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.180+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 shutdown_connections 2023-12-12T09:53:14.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.180+0000 7f6e3b0f3640 1 -- 172.21.15.72:0/962304860 wait complete. 2023-12-12T09:53:14.200 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:14.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.408+0000 7f68d1933640 1 Processor -- start 2023-12-12T09:53:14.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.408+0000 7f68d1933640 1 -- start start 2023-12-12T09:53:14.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68d1933640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68cc0ccf20 0x7f68cc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68d1933640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f68cc0cde30 0x7f68cc0d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68d1933640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68cc0b3e90 con 0x7f68cc144ab0 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68d1933640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc0ce2e0 con 0x7f68cc0cde30 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68d1933640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc0cd6c0 con 0x7f68cc0ccf20 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68caffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f68cc144ab0 0x7f68cc0cb530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40046/0 (socket says 172.21.15.72:40046) 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68caffd640 1 -- 172.21.15.72:0/2803284766 learned_addr learned my addr 172.21.15.72:0/2803284766 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68cb7fe640 1 --2- 172.21.15.72:0/2803284766 >> v2:172.21.15.72:3300/0 conn(0x7f68cc0cde30 0x7f68cc0d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68ca7fc640 1 --2- 172.21.15.72:0/2803284766 >> v2:172.21.15.113:3300/0 conn(0x7f68cc0ccf20 0x7f68cc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.410+0000 7f68c9ffb640 1 -- 172.21.15.72:0/2803284766 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252677645 0 0) 0x7f68cc0b3e90 con 0x7f68cc144ab0 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.411+0000 7f68c9ffb640 1 -- 172.21.15.72:0/2803284766 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68b4003520 con 0x7f68cc144ab0 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.411+0000 7f68cb7fe640 1 -- 172.21.15.72:0/2803284766 >> v2:172.21.15.113:3300/0 conn(0x7f68cc0ccf20 msgr2=0x7f68cc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.411+0000 7f68cb7fe640 1 --2- 172.21.15.72:0/2803284766 >> v2:172.21.15.113:3300/0 conn(0x7f68cc0ccf20 0x7f68cc0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.411+0000 7f68cb7fe640 1 -- 172.21.15.72:0/2803284766 >> v1:172.21.15.72:6789/0 conn(0x7f68cc144ab0 legacy=0x7f68cc0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.411+0000 7f68cb7fe640 1 -- 172.21.15.72:0/2803284766 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc144e90 con 0x7f68cc0cde30 2023-12-12T09:53:14.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.411+0000 7f68ca7fc640 1 --2- 172.21.15.72:0/2803284766 >> v2:172.21.15.113:3300/0 conn(0x7f68cc0ccf20 0x7f68cc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.412+0000 7f68cb7fe640 1 --2- 172.21.15.72:0/2803284766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc0d0670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0b3c50 tx=0x7f68c402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f90304dc877a25 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.412+0000 7f68c9ffb640 1 -- 172.21.15.72:0/2803284766 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c402cbf0 con 0x7f68cc0cde30 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.412+0000 7f68c9ffb640 1 -- 172.21.15.72:0/2803284766 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c402cd50 con 0x7f68cc0cde30 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.412+0000 7f68c9ffb640 1 -- 172.21.15.72:0/2803284766 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4034660 con 0x7f68cc0cde30 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.412+0000 7f68d1933640 1 -- 172.21.15.72:0/2803284766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 msgr2=0x7f68cc0d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.413+0000 7f68d1933640 1 --2- 172.21.15.72:0/2803284766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc0d0670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0b3c50 tx=0x7f68c402be40 comp rx=0 tx=0).stop 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.413+0000 7f68d1933640 1 -- 172.21.15.72:0/2803284766 shutdown_connections 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.413+0000 7f68d1933640 1 --2- 172.21.15.72:0/2803284766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc0d0670 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.413+0000 7f68d1933640 1 --2- 172.21.15.72:0/2803284766 >> v2:172.21.15.113:3300/0 conn(0x7f68cc0ccf20 0x7f68cc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.413+0000 7f68d1933640 1 -- 172.21.15.72:0/2803284766 >> 172.21.15.72:0/2803284766 conn(0x7f68cc0661f0 msgr2=0x7f68cc0665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:14.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.413+0000 7f68d1933640 1 -- 172.21.15.72:0/2803284766 shutdown_connections 2023-12-12T09:53:14.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.413+0000 7f68d1933640 1 -- 172.21.15.72:0/2803284766 wait complete. 2023-12-12T09:53:14.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.414+0000 7f68d1933640 1 Processor -- start 2023-12-12T09:53:14.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.415+0000 7f68d1933640 1 -- start start 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.415+0000 7f68d1933640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0ccf20 0x7f68cc15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68d1933640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68d1933640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68cc144ab0 0x7f68cc13f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68d1933640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc138840 con 0x7f68cc0cde30 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68d1933640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc137f70 con 0x7f68cc0ccf20 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68d1933640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc142500 con 0x7f68cc144ab0 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68ca7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68ca7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc138cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36454/0 (socket says 172.21.15.72:36454) 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68ca7fc640 1 -- 172.21.15.72:0/3186390055 learned_addr learned my addr 172.21.15.72:0/3186390055 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68ca7fc640 1 -- 172.21.15.72:0/3186390055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68cc144ab0 msgr2=0x7f68cc13f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68ca7fc640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68cc144ab0 0x7f68cc13f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68ca7fc640 1 -- 172.21.15.72:0/3186390055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0ccf20 msgr2=0x7f68cc15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68caffd640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0ccf20 0x7f68cc15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68ca7fc640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0ccf20 0x7f68cc15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68ca7fc640 1 -- 172.21.15.72:0/3186390055 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4009a70 con 0x7f68cc0cde30 2023-12-12T09:53:14.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.416+0000 7f68caffd640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0ccf20 0x7f68cc15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:14.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.417+0000 7f68ca7fc640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc138cd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f68c000e760 tx=0x7f68c000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:14.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.417+0000 7f68b3fff640 1 -- 172.21.15.72:0/3186390055 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c0004230 con 0x7f68cc0cde30 2023-12-12T09:53:14.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.417+0000 7f68b3fff640 1 -- 172.21.15.72:0/3186390055 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c0004390 con 0x7f68cc0cde30 2023-12-12T09:53:14.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.417+0000 7f68b3fff640 1 -- 172.21.15.72:0/3186390055 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c000c8f0 con 0x7f68cc0cde30 2023-12-12T09:53:14.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.417+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc1381f0 con 0x7f68cc0cde30 2023-12-12T09:53:14.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.417+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc15ac80 con 0x7f68cc0cde30 2023-12-12T09:53:14.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.418+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc1389f0 con 0x7f68cc0cde30 2023-12-12T09:53:14.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.419+0000 7f68b3fff640 1 -- 172.21.15.72:0/3186390055 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68c000ca50 con 0x7f68cc0cde30 2023-12-12T09:53:14.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.420+0000 7f68b3fff640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f68b806ae40 0x7f68b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.420+0000 7f68b3fff640 1 -- 172.21.15.72:0/3186390055 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f68c00896e0 con 0x7f68cc0cde30 2023-12-12T09:53:14.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.420+0000 7f68caffd640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f68b806ae40 0x7f68b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.421+0000 7f68caffd640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f68b806ae40 0x7f68b806d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0011e0 tx=0x7f68bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:14.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.424+0000 7f68b3fff640 1 -- 172.21.15.72:0/3186390055 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68cc1389f0 con 0x7f68cc0cde30 2023-12-12T09:53:14.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.601+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68cc1389f0 con 0x7f68cc0cde30 2023-12-12T09:53:14.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.602+0000 7f68b3fff640 1 -- 172.21.15.72:0/3186390055 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68cc1389f0 con 0x7f68cc0cde30 2023-12-12T09:53:14.604 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:14.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f68b806ae40 msgr2=0x7f68b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f68b806ae40 0x7f68b806d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0011e0 tx=0x7f68bc009040 comp rx=0 tx=0).stop 2023-12-12T09:53:14.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 msgr2=0x7f68cc138cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc138cd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f68c000e760 tx=0x7f68c000ec20 comp rx=0 tx=0).stop 2023-12-12T09:53:14.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 shutdown_connections 2023-12-12T09:53:14.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f68b806ae40 0x7f68b806d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68cc144ab0 0x7f68cc13f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68cc0cde30 0x7f68cc138cd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 --2- 172.21.15.72:0/3186390055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0ccf20 0x7f68cc15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.604+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 >> 172.21.15.72:0/3186390055 conn(0x7f68cc0661f0 msgr2=0x7f68cc15dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:14.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.605+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 shutdown_connections 2023-12-12T09:53:14.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.605+0000 7f68d1933640 1 -- 172.21.15.72:0/3186390055 wait complete. 2023-12-12T09:53:14.624 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:14.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.827+0000 7fa59def1640 1 Processor -- start 2023-12-12T09:53:14.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.827+0000 7fa59def1640 1 -- start start 2023-12-12T09:53:14.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.828+0000 7fa59def1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5980cb5b0 0x7fa59806c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.828+0000 7fa59def1640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa598070c60 0x7fa598071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.828+0000 7fa59def1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59806c770 con 0x7fa5980cb990 2023-12-12T09:53:14.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.828+0000 7fa59def1640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59806ca50 con 0x7fa598070c60 2023-12-12T09:53:14.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.828+0000 7fa59def1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59806cd30 con 0x7fa5980cb5b0 2023-12-12T09:53:14.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59d6f0640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa598070c60 0x7fa598071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59d6f0640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa598070c60 0x7fa598071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36458/0 (socket says 172.21.15.72:36458) 2023-12-12T09:53:14.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59d6f0640 1 -- 172.21.15.72:0/1055012132 learned_addr learned my addr 172.21.15.72:0/1055012132 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:14.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59ceef640 1 --2- 172.21.15.72:0/1055012132 >> v2:172.21.15.113:3300/0 conn(0x7fa5980cb5b0 0x7fa59806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59d6f0640 1 -- 172.21.15.72:0/1055012132 >> v2:172.21.15.113:3300/0 conn(0x7fa5980cb5b0 msgr2=0x7fa59806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:14.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59d6f0640 1 --2- 172.21.15.72:0/1055012132 >> v2:172.21.15.113:3300/0 conn(0x7fa5980cb5b0 0x7fa59806c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59d6f0640 1 -- 172.21.15.72:0/1055012132 >> v1:172.21.15.113:6789/0 conn(0x7fa5980cb990 legacy=0x7fa59815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:14.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59d6f0640 1 -- 172.21.15.72:0/1055012132 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59806d010 con 0x7fa598070c60 2023-12-12T09:53:14.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.829+0000 7fa59ceef640 1 --2- 172.21.15.72:0/1055012132 >> v2:172.21.15.113:3300/0 conn(0x7fa5980cb5b0 0x7fa59806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:14.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.830+0000 7fa59d6f0640 1 --2- 172.21.15.72:0/1055012132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa598070c60 0x7fa598071110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5980cfba0 tx=0x7fa59002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f4b70d6a692679 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:14.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa58f7fe640 1 -- 172.21.15.72:0/1055012132 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59002ca80 con 0x7fa598070c60 2023-12-12T09:53:14.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa58f7fe640 1 -- 172.21.15.72:0/1055012132 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59002cbe0 con 0x7fa598070c60 2023-12-12T09:53:14.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa58f7fe640 1 -- 172.21.15.72:0/1055012132 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5900343f0 con 0x7fa598070c60 2023-12-12T09:53:14.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa59def1640 1 -- 172.21.15.72:0/1055012132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa598070c60 msgr2=0x7fa598071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa59def1640 1 --2- 172.21.15.72:0/1055012132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa598070c60 0x7fa598071110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5980cfba0 tx=0x7fa59002bee0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa59def1640 1 -- 172.21.15.72:0/1055012132 shutdown_connections 2023-12-12T09:53:14.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa59def1640 1 --2- 172.21.15.72:0/1055012132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa598070c60 0x7fa598071110 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa59def1640 1 --2- 172.21.15.72:0/1055012132 >> v2:172.21.15.113:3300/0 conn(0x7fa5980cb5b0 0x7fa59806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa59def1640 1 -- 172.21.15.72:0/1055012132 >> 172.21.15.72:0/1055012132 conn(0x7fa5980b0940 msgr2=0x7fa5980b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:14.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa59def1640 1 -- 172.21.15.72:0/1055012132 shutdown_connections 2023-12-12T09:53:14.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.831+0000 7fa59def1640 1 -- 172.21.15.72:0/1055012132 wait complete. 2023-12-12T09:53:14.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.833+0000 7fa59def1640 1 Processor -- start 2023-12-12T09:53:14.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.833+0000 7fa59def1640 1 -- start start 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59def1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa598070c60 0x7fa5980cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59def1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5980cb5b0 0x7fa59814c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59def1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5980cb990 0x7fa5980cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59def1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980cfb00 con 0x7fa5980cb990 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59def1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980b28f0 con 0x7fa598070c60 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59def1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980b2bd0 con 0x7fa5980cb5b0 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59d6f0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5980cb990 0x7fa5980cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59d6f0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5980cb990 0x7fa5980cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36460/0 (socket says 172.21.15.72:36460) 2023-12-12T09:53:14.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.834+0000 7fa59d6f0640 1 -- 172.21.15.72:0/818450294 learned_addr learned my addr 172.21.15.72:0/818450294 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59ceef640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa598070c60 0x7fa5980cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59d6f0640 1 -- 172.21.15.72:0/818450294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5980cb5b0 msgr2=0x7fa59814c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59d6f0640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5980cb5b0 0x7fa59814c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59d6f0640 1 -- 172.21.15.72:0/818450294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa598070c60 msgr2=0x7fa5980cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59d6f0640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa598070c60 0x7fa5980cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59d6f0640 1 -- 172.21.15.72:0/818450294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa590009b30 con 0x7fa5980cb990 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59d6f0640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5980cb990 0x7fa5980cd1a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa590036040 tx=0x7fa590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa58d7fa640 1 -- 172.21.15.72:0/818450294 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59003e450 con 0x7fa5980cb990 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa58d7fa640 1 -- 172.21.15.72:0/818450294 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59003e5d0 con 0x7fa5980cb990 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5980b2eb0 con 0x7fa5980cb990 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa58d7fa640 1 -- 172.21.15.72:0/818450294 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59003e980 con 0x7fa5980cb990 2023-12-12T09:53:14.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.835+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5980b36b0 con 0x7fa5980cb990 2023-12-12T09:53:14.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.836+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa598067910 con 0x7fa5980cb990 2023-12-12T09:53:14.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.837+0000 7fa58d7fa640 1 -- 172.21.15.72:0/818450294 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa59002cd50 con 0x7fa5980cb990 2023-12-12T09:53:14.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.838+0000 7fa58d7fa640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa58406ae40 0x7fa58406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:14.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.838+0000 7fa58d7fa640 1 -- 172.21.15.72:0/818450294 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa5900ac510 con 0x7fa5980cb990 2023-12-12T09:53:14.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.839+0000 7fa59ceef640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa58406ae40 0x7fa58406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:14.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.839+0000 7fa59ceef640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa58406ae40 0x7fa58406d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa594008850 tx=0x7fa594008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:14.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:14.843+0000 7fa58d7fa640 1 -- 172.21.15.72:0/818450294 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa590077290 con 0x7fa5980cb990 2023-12-12T09:53:15.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.030+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa5980b33d0 con 0x7fa5980cb990 2023-12-12T09:53:15.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.031+0000 7fa58d7fa640 1 -- 172.21.15.72:0/818450294 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa59007a8b0 con 0x7fa5980cb990 2023-12-12T09:53:15.033 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:15.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.033+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa58406ae40 msgr2=0x7fa58406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.033+0000 7fa59def1640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa58406ae40 0x7fa58406d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa594008850 tx=0x7fa594008040 comp rx=0 tx=0).stop 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5980cb990 msgr2=0x7fa5980cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5980cb990 0x7fa5980cd1a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa590036040 tx=0x7fa590004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 shutdown_connections 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa58406ae40 0x7fa58406d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5980cb990 0x7fa5980cd1a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5980cb5b0 0x7fa59814c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 --2- 172.21.15.72:0/818450294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa598070c60 0x7fa5980cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 >> 172.21.15.72:0/818450294 conn(0x7fa5980b0940 msgr2=0x7fa5981549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 shutdown_connections 2023-12-12T09:53:15.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.034+0000 7fa59def1640 1 -- 172.21.15.72:0/818450294 wait complete. 2023-12-12T09:53:15.056 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_strays.TestStrays.test_dir_deletion' 2023-12-12T09:53:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.252+0000 7f45dc501640 1 Processor -- start 2023-12-12T09:53:15.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.252+0000 7f45dc501640 1 -- start start 2023-12-12T09:53:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.254+0000 7f45dc501640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45d415d580 0x7f45d40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.254+0000 7f45dc501640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f45d406c690 0x7f45d40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.254+0000 7f45dc501640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d40b07f0 con 0x7f45d40cfc50 2023-12-12T09:53:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.254+0000 7f45dc501640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d40d0620 con 0x7f45d406c690 2023-12-12T09:53:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.254+0000 7f45dc501640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d415d960 con 0x7f45d415d580 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.254+0000 7f45daa9e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f45d406c690 0x7f45d40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.254+0000 7f45daa9e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f45d406c690 0x7f45d40d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41778/0 (socket says 172.21.15.72:41778) 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.254+0000 7f45daa9e640 1 -- 172.21.15.72:0/2525170108 learned_addr learned my addr 172.21.15.72:0/2525170108 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.255+0000 7f45da29d640 1 --2- 172.21.15.72:0/2525170108 >> v2:172.21.15.113:3300/0 conn(0x7f45d415d580 0x7f45d40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.255+0000 7f45daa9e640 1 -- 172.21.15.72:0/2525170108 >> v2:172.21.15.113:3300/0 conn(0x7f45d415d580 msgr2=0x7f45d40adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.255+0000 7f45daa9e640 1 --2- 172.21.15.72:0/2525170108 >> v2:172.21.15.113:3300/0 conn(0x7f45d415d580 0x7f45d40adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.255+0000 7f45daa9e640 1 -- 172.21.15.72:0/2525170108 >> v1:172.21.15.72:6789/0 conn(0x7f45d40cfc50 legacy=0x7f45d406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.255+0000 7f45d929b640 1 -- 172.21.15.72:0/2525170108 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728750748 0 0) 0x7f45d40b07f0 con 0x7f45d40cfc50 2023-12-12T09:53:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.255+0000 7f45daa9e640 1 -- 172.21.15.72:0/2525170108 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d40b0a40 con 0x7f45d406c690 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.256+0000 7f45daa9e640 1 --2- 172.21.15.72:0/2525170108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40d00e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f45d4070e10 tx=0x7f45cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e9d830e93d5029 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.256+0000 7f45d929b640 1 -- 172.21.15.72:0/2525170108 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc030860 con 0x7f45d406c690 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.256+0000 7f45d929b640 1 -- 172.21.15.72:0/2525170108 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc0309c0 con 0x7f45d406c690 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.256+0000 7f45d929b640 1 -- 172.21.15.72:0/2525170108 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc030d50 con 0x7f45d406c690 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.257+0000 7f45dc501640 1 -- 172.21.15.72:0/2525170108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 msgr2=0x7f45d40d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.257+0000 7f45dc501640 1 --2- 172.21.15.72:0/2525170108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40d00e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f45d4070e10 tx=0x7f45cc02b9a0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.257+0000 7f45dc501640 1 -- 172.21.15.72:0/2525170108 shutdown_connections 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.257+0000 7f45dc501640 1 --2- 172.21.15.72:0/2525170108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40d00e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.257+0000 7f45dc501640 1 --2- 172.21.15.72:0/2525170108 >> v2:172.21.15.113:3300/0 conn(0x7f45d415d580 0x7f45d40adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.257+0000 7f45dc501640 1 -- 172.21.15.72:0/2525170108 >> 172.21.15.72:0/2525170108 conn(0x7f45d40d0e90 msgr2=0x7f45d40d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.257+0000 7f45dc501640 1 -- 172.21.15.72:0/2525170108 shutdown_connections 2023-12-12T09:53:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.257+0000 7f45dc501640 1 -- 172.21.15.72:0/2525170108 wait complete. 2023-12-12T09:53:15.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.259+0000 7f45dc501640 1 Processor -- start 2023-12-12T09:53:15.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.259+0000 7f45dc501640 1 -- start start 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45dc501640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45dc501640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45d40cfc50 0x7f45d415fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45dc501640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d415d580 0x7f45d40b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45dc501640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4160040 con 0x7f45d406c690 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45dc501640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d40b4330 con 0x7f45d415d580 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45dc501640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d40b4610 con 0x7f45d40cfc50 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45da29d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45da29d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41790/0 (socket says 172.21.15.72:41790) 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45da29d640 1 -- 172.21.15.72:0/3310123163 learned_addr learned my addr 172.21.15.72:0/3310123163 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45da29d640 1 -- 172.21.15.72:0/3310123163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45d40cfc50 msgr2=0x7f45d415fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45da29d640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45d40cfc50 0x7f45d415fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45da29d640 1 -- 172.21.15.72:0/3310123163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d415d580 msgr2=0x7f45d40b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45da29d640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d415d580 0x7f45d40b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.260+0000 7f45da29d640 1 -- 172.21.15.72:0/3310123163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc0094d0 con 0x7f45d406c690 2023-12-12T09:53:15.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.261+0000 7f45da29d640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40b3ce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f45c400d5f0 tx=0x7f45c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:15.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.261+0000 7f45caffd640 1 -- 172.21.15.72:0/3310123163 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c40125f0 con 0x7f45d406c690 2023-12-12T09:53:15.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.261+0000 7f45caffd640 1 -- 172.21.15.72:0/3310123163 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c4012750 con 0x7f45d406c690 2023-12-12T09:53:15.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.261+0000 7f45caffd640 1 -- 172.21.15.72:0/3310123163 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4012b00 con 0x7f45d406c690 2023-12-12T09:53:15.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.261+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d40b48f0 con 0x7f45d406c690 2023-12-12T09:53:15.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.261+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d40b4eb0 con 0x7f45d406c690 2023-12-12T09:53:15.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.262+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d40745d0 con 0x7f45d406c690 2023-12-12T09:53:15.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.263+0000 7f45caffd640 1 -- 172.21.15.72:0/3310123163 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45c4012c60 con 0x7f45d406c690 2023-12-12T09:53:15.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.264+0000 7f45caffd640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f45c006ae40 0x7f45c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:15.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.264+0000 7f45caffd640 1 -- 172.21.15.72:0/3310123163 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f45c4089f40 con 0x7f45d406c690 2023-12-12T09:53:15.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.264+0000 7f45d9a9c640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f45c006ae40 0x7f45c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:15.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.265+0000 7f45d9a9c640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f45c006ae40 0x7f45c006d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f45d4159630 tx=0x7f45d0007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:15.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.268+0000 7f45caffd640 1 -- 172.21.15.72:0/3310123163 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45c4054b90 con 0x7f45d406c690 2023-12-12T09:53:15.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:15.526+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_strays.TestStrays.test_dir_deletion"]} v 0) v1 -- 0x7f45d406bd60 con 0x7f45d406c690 2023-12-12T09:53:16.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.503+0000 7f45caffd640 1 -- 172.21.15.72:0/3310123163 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_strays.TestStrays.test_dir_deletion"]}]=0 v80) v1 ==== 135+0+0 (secure 0 0 0) 0x7f45c40581b0 con 0x7f45d406c690 2023-12-12T09:53:16.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f45c006ae40 msgr2=0x7f45c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:16.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f45c006ae40 0x7f45c006d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f45d4159630 tx=0x7f45d0007c90 comp rx=0 tx=0).stop 2023-12-12T09:53:16.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 msgr2=0x7f45d40b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:16.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40b3ce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f45c400d5f0 tx=0x7f45c400dab0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 shutdown_connections 2023-12-12T09:53:16.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f45c006ae40 0x7f45c006d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d415d580 0x7f45d40b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45d40cfc50 0x7f45d415fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 --2- 172.21.15.72:0/3310123163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45d406c690 0x7f45d40b3ce0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 >> 172.21.15.72:0/3310123163 conn(0x7f45d40d0e90 msgr2=0x7f45d406bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:16.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.506+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 shutdown_connections 2023-12-12T09:53:16.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.507+0000 7f45dc501640 1 -- 172.21.15.72:0/3310123163 wait complete. 2023-12-12T09:53:16.528 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-12T09:53:16.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.741+0000 7f0f3a690640 1 Processor -- start 2023-12-12T09:53:16.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.742+0000 7f0f3a690640 1 -- start start 2023-12-12T09:53:16.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.743+0000 7f0f3a690640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f3415af50 0x7f0f3406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:16.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.743+0000 7f0f3a690640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f3415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:16.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.743+0000 7f0f3a690640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f34071cd0 con 0x7f0f3406a460 2023-12-12T09:53:16.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.743+0000 7f0f3a690640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f340717e0 con 0x7f0f3415af50 2023-12-12T09:53:16.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.743+0000 7f0f3a690640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3406c6b0 con 0x7f0f340ce7e0 2023-12-12T09:53:16.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.744+0000 7f0f337fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f3415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:16.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.744+0000 7f0f33fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f3415af50 0x7f0f3406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:16.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.744+0000 7f0f33fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f3415af50 0x7f0f3406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:54624/0 (socket says 172.21.15.72:54624) 2023-12-12T09:53:16.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.744+0000 7f0f337fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f3415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:39150/0 (socket says 172.21.15.72:39150) 2023-12-12T09:53:16.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.744+0000 7f0f33fff640 1 -- 172.21.15.72:0/1256776134 learned_addr learned my addr 172.21.15.72:0/1256776134 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:16.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.745+0000 7f0f32ffd640 1 -- 172.21.15.72:0/1256776134 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804737909 0 0) 0x7f0f34071cd0 con 0x7f0f3406a460 2023-12-12T09:53:16.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.745+0000 7f0f32ffd640 1 -- 172.21.15.72:0/1256776134 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f20003520 con 0x7f0f3406a460 2023-12-12T09:53:16.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.745+0000 7f0f337fe640 1 -- 172.21.15.72:0/1256776134 >> v2:172.21.15.113:3300/0 conn(0x7f0f3415af50 msgr2=0x7f0f3406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:16.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.745+0000 7f0f337fe640 1 --2- 172.21.15.72:0/1256776134 >> v2:172.21.15.113:3300/0 conn(0x7f0f3415af50 0x7f0f3406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.745+0000 7f0f337fe640 1 -- 172.21.15.72:0/1256776134 >> v1:172.21.15.113:6789/0 conn(0x7f0f3406a460 legacy=0x7f0f340ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:16.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.745+0000 7f0f337fe640 1 -- 172.21.15.72:0/1256776134 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f340719f0 con 0x7f0f340ce7e0 2023-12-12T09:53:16.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.746+0000 7f0f337fe640 1 --2- 172.21.15.72:0/1256776134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f3415ec00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0097a0 tx=0x7f0f1c02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c71975db94bee89 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:16.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.747+0000 7f0f32ffd640 1 -- 172.21.15.72:0/1256776134 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c02f050 con 0x7f0f340ce7e0 2023-12-12T09:53:16.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.747+0000 7f0f32ffd640 1 -- 172.21.15.72:0/1256776134 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c031700 con 0x7f0f340ce7e0 2023-12-12T09:53:16.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.747+0000 7f0f32ffd640 1 -- 172.21.15.72:0/1256776134 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c031a90 con 0x7f0f340ce7e0 2023-12-12T09:53:16.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.748+0000 7f0f3a690640 1 -- 172.21.15.72:0/1256776134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 msgr2=0x7f0f3415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:16.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.748+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1256776134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f3415ec00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0097a0 tx=0x7f0f1c02b7a0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.748+0000 7f0f3a690640 1 -- 172.21.15.72:0/1256776134 shutdown_connections 2023-12-12T09:53:16.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.748+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1256776134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f3415ec00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.748+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1256776134 >> v2:172.21.15.113:3300/0 conn(0x7f0f3415af50 0x7f0f3406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.748+0000 7f0f3a690640 1 -- 172.21.15.72:0/1256776134 >> 172.21.15.72:0/1256776134 conn(0x7f0f340cee70 msgr2=0x7f0f340cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:16.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.748+0000 7f0f3a690640 1 -- 172.21.15.72:0/1256776134 shutdown_connections 2023-12-12T09:53:16.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.748+0000 7f0f3a690640 1 -- 172.21.15.72:0/1256776134 wait complete. 2023-12-12T09:53:16.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.750+0000 7f0f3a690640 1 Processor -- start 2023-12-12T09:53:16.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.750+0000 7f0f3a690640 1 -- start start 2023-12-12T09:53:16.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f3a690640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f3406a460 0x7f0f340c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f3a690640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f340af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f3a690640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f3415af50 0x7f0f3414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f3a690640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340c4520 con 0x7f0f3406a460 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f3a690640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340c4800 con 0x7f0f3415af50 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f3a690640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340c4ae0 con 0x7f0f340ce7e0 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f3406a460 0x7f0f340c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f3406a460 0x7f0f340c3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41794/0 (socket says 172.21.15.72:41794) 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.751+0000 7f0f33fff640 1 -- 172.21.15.72:0/1813573505 learned_addr learned my addr 172.21.15.72:0/1813573505 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f337fe640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f340af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f38c2d640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f3415af50 0x7f0f3414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f33fff640 1 -- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 msgr2=0x7f0f340af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f33fff640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f340af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f33fff640 1 -- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f3415af50 msgr2=0x7f0f3414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:16.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f33fff640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f3415af50 0x7f0f3414ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f33fff640 1 -- 172.21.15.72:0/1813573505 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c009400 con 0x7f0f3406a460 2023-12-12T09:53:16.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f38c2d640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f3415af50 0x7f0f3414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:16.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f33fff640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f3406a460 0x7f0f340c3e90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400efc0 tx=0x7f0f2400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:16.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f30ff9640 1 -- 172.21.15.72:0/1813573505 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400cda0 con 0x7f0f3406a460 2023-12-12T09:53:16.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f30ff9640 1 -- 172.21.15.72:0/1813573505 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2400f040 con 0x7f0f3406a460 2023-12-12T09:53:16.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f30ff9640 1 -- 172.21.15.72:0/1813573505 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24014680 con 0x7f0f3406a460 2023-12-12T09:53:16.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.752+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f340c4dc0 con 0x7f0f3406a460 2023-12-12T09:53:16.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.753+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f341504c0 con 0x7f0f3406a460 2023-12-12T09:53:16.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.754+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f340cdfc0 con 0x7f0f3406a460 2023-12-12T09:53:16.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.755+0000 7f0f30ff9640 1 -- 172.21.15.72:0/1813573505 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f240147e0 con 0x7f0f3406a460 2023-12-12T09:53:16.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.755+0000 7f0f30ff9640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0f1406ae40 0x7f0f1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:16.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.755+0000 7f0f30ff9640 1 -- 172.21.15.72:0/1813573505 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0f2408a1b0 con 0x7f0f3406a460 2023-12-12T09:53:16.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.756+0000 7f0f337fe640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0f1406ae40 0x7f0f1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:16.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.756+0000 7f0f337fe640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0f1406ae40 0x7f0f1406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c035000 tx=0x7f0f1c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:16.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.760+0000 7f0f30ff9640 1 -- 172.21.15.72:0/1813573505 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f24054e90 con 0x7f0f3406a460 2023-12-12T09:53:16.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.950+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0f34074450 con 0x7f0f3406a460 2023-12-12T09:53:16.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.958+0000 7f0f30ff9640 1 -- 172.21.15.72:0/1813573505 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0f240584b0 con 0x7f0f3406a460 2023-12-12T09:53:16.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0f1406ae40 msgr2=0x7f0f1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:16.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0f1406ae40 0x7f0f1406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c035000 tx=0x7f0f1c008030 comp rx=0 tx=0).stop 2023-12-12T09:53:16.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f3406a460 msgr2=0x7f0f340c3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:16.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f3406a460 0x7f0f340c3e90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400efc0 tx=0x7f0f2400c490 comp rx=0 tx=0).stop 2023-12-12T09:53:16.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 shutdown_connections 2023-12-12T09:53:16.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0f1406ae40 0x7f0f1406d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f3415af50 0x7f0f3414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f340ce7e0 0x7f0f340af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 --2- 172.21.15.72:0/1813573505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f3406a460 0x7f0f340c3e90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:16.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.960+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 >> 172.21.15.72:0/1813573505 conn(0x7f0f340cee70 msgr2=0x7f0f340700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:16.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.961+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 shutdown_connections 2023-12-12T09:53:16.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:16.961+0000 7f0f3a690640 1 -- 172.21.15.72:0/1813573505 wait complete. 2023-12-12T09:53:16.981 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-12T09:53:16.981 DEBUG:teuthology.orchestra.run.smithi072:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-12T09:53:17.094 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-12T09:53:17.213 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-12T09:53:17.213 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:53:17.213 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:17.274 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:53:17.275 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:17.330 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:53:17.330 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:17.385 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.597+0000 7fc17cfce640 1 Processor -- start 2023-12-12T09:53:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.597+0000 7fc17cfce640 1 -- start start 2023-12-12T09:53:17.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.599+0000 7fc17cfce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc178143e00 0x7fc1780b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:17.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.599+0000 7fc17cfce640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1780b3bb0 con 0x7fc178072180 2023-12-12T09:53:17.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.599+0000 7fc17cfce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1781441e0 con 0x7fc17806c410 2023-12-12T09:53:17.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.599+0000 7fc17cfce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc178072680 con 0x7fc178143e00 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.599+0000 7fc175d9b640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc178072180 0x7fc1780b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42140/0 (socket says 172.21.15.72:42140) 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc175d9b640 1 -- 172.21.15.72:0/3279253687 learned_addr learned my addr 172.21.15.72:0/3279253687 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17659c640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc178143e00 0x7fc1780b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245138981 0 0) 0x7fc1780b3bb0 con 0x7fc178072180 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc164003520 con 0x7fc178072180 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738461415 0 0) 0x7fc164003520 con 0x7fc178072180 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc178143e00 msgr2=0x7fc1780b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc178143e00 0x7fc1780b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 >> v1:172.21.15.113:6789/0 conn(0x7fc17806c410 legacy=0x7fc17815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780b1e30 con 0x7fc178072180 2023-12-12T09:53:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fc160001670 con 0x7fc178072180 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 >> v1:172.21.15.72:6789/0 conn(0x7fc178072180 legacy=0x7fc1780b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.600+0000 7fc17559a640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc164003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17559a640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640047c0 0x7fc164004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17559a640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc178143e00 0x7fc164005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc178072680 con 0x7fc1640039a0 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16400ba30 con 0x7fc1640047c0 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16400bbf0 con 0x7fc178143e00 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17659c640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc164003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17659c640 1 -- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc178143e00 msgr2=0x7fc164005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17659c640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc178143e00 0x7fc164005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17659c640 1 -- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640047c0 msgr2=0x7fc164004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17659c640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1640047c0 0x7fc164004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.601+0000 7fc17659c640 1 -- 172.21.15.72:0/3279253687 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164003520 con 0x7fc1640039a0 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.602+0000 7fc17659c640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc164003dc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc168009a90 tx=0x7fc1640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23879ac2a964eb77 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.602+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16802c6b0 con 0x7fc1640039a0 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.602+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16802c810 con 0x7fc1640039a0 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.602+0000 7fc17559a640 1 -- 172.21.15.72:0/3279253687 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16802cba0 con 0x7fc1640039a0 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.603+0000 7fc17cfce640 1 -- 172.21.15.72:0/3279253687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 msgr2=0x7fc164003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.603+0000 7fc17cfce640 1 --2- 172.21.15.72:0/3279253687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc164003dc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc168009a90 tx=0x7fc1640057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.603+0000 7fc176d9d640 1 -- 172.21.15.72:0/3279253687 reap_dead start 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.603+0000 7fc17cfce640 1 -- 172.21.15.72:0/3279253687 shutdown_connections 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.603+0000 7fc17cfce640 1 -- 172.21.15.72:0/3279253687 >> 172.21.15.72:0/3279253687 conn(0x7fc1780b0a90 msgr2=0x7fc1780b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.603+0000 7fc17cfce640 1 -- 172.21.15.72:0/3279253687 shutdown_connections 2023-12-12T09:53:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.603+0000 7fc17cfce640 1 -- 172.21.15.72:0/3279253687 wait complete. 2023-12-12T09:53:17.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.605+0000 7fc17cfce640 1 Processor -- start 2023-12-12T09:53:17.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.605+0000 7fc17cfce640 1 -- start start 2023-12-12T09:53:17.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17cfce640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc178138440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:17.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17cfce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1781599e0 0x7fc17813a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:17.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17cfce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc17813ad30 0x7fc178140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17cfce640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17813b1b0 con 0x7fc1640039a0 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17cfce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc178138a50 con 0x7fc1781599e0 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17cfce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17813b390 con 0x7fc17813ad30 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc178138440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc178138440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41822/0 (socket says 172.21.15.72:41822) 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.606+0000 7fc17659c640 1 -- 172.21.15.72:0/3607865017 learned_addr learned my addr 172.21.15.72:0/3607865017 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc176d9d640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc17813ad30 0x7fc178140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc17659c640 1 -- 172.21.15.72:0/3607865017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc17813ad30 msgr2=0x7fc178140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc175d9b640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1781599e0 0x7fc17813a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc17659c640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc17813ad30 0x7fc178140f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc17659c640 1 -- 172.21.15.72:0/3607865017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1781599e0 msgr2=0x7fc17813a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc17659c640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1781599e0 0x7fc17813a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc17659c640 1 -- 172.21.15.72:0/3607865017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17813b670 con 0x7fc1640039a0 2023-12-12T09:53:17.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc17659c640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc178138440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc16802c250 tx=0x7fc168004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:17.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc16effd640 1 -- 172.21.15.72:0/3607865017 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168003810 con 0x7fc1640039a0 2023-12-12T09:53:17.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc16effd640 1 -- 172.21.15.72:0/3607865017 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc168003970 con 0x7fc1640039a0 2023-12-12T09:53:17.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17815a1c0 con 0x7fc1640039a0 2023-12-12T09:53:17.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.607+0000 7fc16effd640 1 -- 172.21.15.72:0/3607865017 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168003ce0 con 0x7fc1640039a0 2023-12-12T09:53:17.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.608+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc178141640 con 0x7fc1640039a0 2023-12-12T09:53:17.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.609+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1780745d0 con 0x7fc1640039a0 2023-12-12T09:53:17.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.610+0000 7fc16effd640 1 -- 172.21.15.72:0/3607865017 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc168040450 con 0x7fc1640039a0 2023-12-12T09:53:17.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.610+0000 7fc16effd640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc15806ae40 0x7fc15806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:17.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.610+0000 7fc16effd640 1 -- 172.21.15.72:0/3607865017 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc1680ab790 con 0x7fc1640039a0 2023-12-12T09:53:17.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.611+0000 7fc175d9b640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc15806ae40 0x7fc15806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:17.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.611+0000 7fc175d9b640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc15806ae40 0x7fc15806d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc1600011e0 tx=0x7fc160009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:17.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.615+0000 7fc16effd640 1 -- 172.21.15.72:0/3607865017 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc168076480 con 0x7fc1640039a0 2023-12-12T09:53:17.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.805+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc17813b800 con 0x7fc1640039a0 2023-12-12T09:53:17.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.807+0000 7fc16effd640 1 -- 172.21.15.72:0/3607865017 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7fc1680044e0 con 0x7fc1640039a0 2023-12-12T09:53:17.809 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:17.809 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.809+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc15806ae40 msgr2=0x7fc15806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.809+0000 7fc17cfce640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc15806ae40 0x7fc15806d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc1600011e0 tx=0x7fc160009040 comp rx=0 tx=0).stop 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.809+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 msgr2=0x7fc178138440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.809+0000 7fc17cfce640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc178138440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc16802c250 tx=0x7fc168004730 comp rx=0 tx=0).stop 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.810+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 shutdown_connections 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.810+0000 7fc17cfce640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc15806ae40 0x7fc15806d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.810+0000 7fc17cfce640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc17813ad30 0x7fc178140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.810+0000 7fc17cfce640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1781599e0 0x7fc17813a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.810+0000 7fc17cfce640 1 --2- 172.21.15.72:0/3607865017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1640039a0 0x7fc178138440 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.810+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 >> 172.21.15.72:0/3607865017 conn(0x7fc1780b0a90 msgr2=0x7fc1780af880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:17.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.810+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 shutdown_connections 2023-12-12T09:53:17.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:17.810+0000 7fc17cfce640 1 -- 172.21.15.72:0/3607865017 wait complete. 2023-12-12T09:53:17.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:17.835 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:18.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.052+0000 7fb62ea1e640 1 Processor -- start 2023-12-12T09:53:18.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.052+0000 7fb62ea1e640 1 -- start start 2023-12-12T09:53:18.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb62ea1e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62815a1e0 0x7fb62806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb62ea1e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb62806ab70 0x7fb62815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb62ea1e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6280cdaa0 con 0x7fb6280ae300 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb62ea1e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6280cdd80 con 0x7fb62806ab70 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb62ea1e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280ce060 con 0x7fb62815a1e0 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb62cfbb640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb62806ab70 0x7fb62815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb62cfbb640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb62806ab70 0x7fb62815dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41832/0 (socket says 172.21.15.72:41832) 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb62cfbb640 1 -- 172.21.15.72:0/2384925758 learned_addr learned my addr 172.21.15.72:0/2384925758 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.054+0000 7fb627fff640 1 --2- 172.21.15.72:0/2384925758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62815a1e0 0x7fb62806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.055+0000 7fb62cfbb640 1 -- 172.21.15.72:0/2384925758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62815a1e0 msgr2=0x7fb62806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.055+0000 7fb62cfbb640 1 --2- 172.21.15.72:0/2384925758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62815a1e0 0x7fb62806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.055+0000 7fb62cfbb640 1 -- 172.21.15.72:0/2384925758 >> v1:172.21.15.113:6789/0 conn(0x7fb6280ae300 legacy=0x7fb62806a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:18.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.055+0000 7fb62cfbb640 1 -- 172.21.15.72:0/2384925758 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280ce340 con 0x7fb62806ab70 2023-12-12T09:53:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.056+0000 7fb62cfbb640 1 --2- 172.21.15.72:0/2384925758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb62806ab70 0x7fb62815dc60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb620009e50 tx=0x7fb62002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f535efa37b3bbd6c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.056+0000 7fb6277fe640 1 -- 172.21.15.72:0/2384925758 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62002ca70 con 0x7fb62806ab70 2023-12-12T09:53:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.056+0000 7fb6277fe640 1 -- 172.21.15.72:0/2384925758 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62002cbd0 con 0x7fb62806ab70 2023-12-12T09:53:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.056+0000 7fb6277fe640 1 -- 172.21.15.72:0/2384925758 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200343f0 con 0x7fb62806ab70 2023-12-12T09:53:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.057+0000 7fb62ea1e640 1 -- 172.21.15.72:0/2384925758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb62806ab70 msgr2=0x7fb62815dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.057+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/2384925758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb62806ab70 0x7fb62815dc60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb620009e50 tx=0x7fb62002bed0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.057+0000 7fb62ea1e640 1 -- 172.21.15.72:0/2384925758 shutdown_connections 2023-12-12T09:53:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.057+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/2384925758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb62806ab70 0x7fb62815dc60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.057+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/2384925758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62815a1e0 0x7fb62806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.057+0000 7fb62ea1e640 1 -- 172.21.15.72:0/2384925758 >> 172.21.15.72:0/2384925758 conn(0x7fb6280cee70 msgr2=0x7fb6280cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:18.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.057+0000 7fb62ea1e640 1 -- 172.21.15.72:0/2384925758 shutdown_connections 2023-12-12T09:53:18.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.057+0000 7fb62ea1e640 1 -- 172.21.15.72:0/2384925758 wait complete. 2023-12-12T09:53:18.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.059+0000 7fb62ea1e640 1 Processor -- start 2023-12-12T09:53:18.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.059+0000 7fb62ea1e640 1 -- start start 2023-12-12T09:53:18.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb62ea1e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62806ab70 0x7fb62814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb62ea1e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6280ae300 0x7fb6280c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb62ea1e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb62815a1e0 0x7fb6280ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb62ea1e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62814f6e0 con 0x7fb6280ae300 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb62ea1e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280ac3f0 con 0x7fb62815a1e0 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb62ea1e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280ac6d0 con 0x7fb62806ab70 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6280ae300 0x7fb6280c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6280ae300 0x7fb6280c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41842/0 (socket says 172.21.15.72:41842) 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb61f7fe640 1 -- 172.21.15.72:0/967965393 learned_addr learned my addr 172.21.15.72:0/967965393 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb627fff640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62806ab70 0x7fb62814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb62cfbb640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb62815a1e0 0x7fb6280ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb61f7fe640 1 -- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62806ab70 msgr2=0x7fb62814f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb61f7fe640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62806ab70 0x7fb62814f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb61f7fe640 1 -- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb62815a1e0 msgr2=0x7fb6280ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb61f7fe640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb62815a1e0 0x7fb6280ad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.060+0000 7fb61f7fe640 1 -- 172.21.15.72:0/967965393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb620009b30 con 0x7fb6280ae300 2023-12-12T09:53:18.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.061+0000 7fb627fff640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62806ab70 0x7fb62814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:18.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.061+0000 7fb61f7fe640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6280ae300 0x7fb6280c27c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb61000c450 tx=0x7fb61000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.061+0000 7fb6257fa640 1 -- 172.21.15.72:0/967965393 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6100052d0 con 0x7fb6280ae300 2023-12-12T09:53:18.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.061+0000 7fb6257fa640 1 -- 172.21.15.72:0/967965393 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb610005430 con 0x7fb6280ae300 2023-12-12T09:53:18.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.061+0000 7fb6257fa640 1 -- 172.21.15.72:0/967965393 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610012870 con 0x7fb6280ae300 2023-12-12T09:53:18.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.061+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6280ac9b0 con 0x7fb6280ae300 2023-12-12T09:53:18.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.061+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6280acf70 con 0x7fb6280ae300 2023-12-12T09:53:18.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.062+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb628074450 con 0x7fb6280ae300 2023-12-12T09:53:18.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.063+0000 7fb6257fa640 1 -- 172.21.15.72:0/967965393 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6100129d0 con 0x7fb6280ae300 2023-12-12T09:53:18.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.064+0000 7fb6257fa640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb60806ae40 0x7fb60806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.064+0000 7fb6257fa640 1 -- 172.21.15.72:0/967965393 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb610089520 con 0x7fb6280ae300 2023-12-12T09:53:18.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.065+0000 7fb627fff640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb60806ae40 0x7fb60806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.065+0000 7fb627fff640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb60806ae40 0x7fb60806d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb618006fd0 tx=0x7fb618008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.069+0000 7fb6257fa640 1 -- 172.21.15.72:0/967965393 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb610054200 con 0x7fb6280ae300 2023-12-12T09:53:18.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.256+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6280c9b60 con 0x7fb6280ae300 2023-12-12T09:53:18.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.257+0000 7fb6257fa640 1 -- 172.21.15.72:0/967965393 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb610057820 con 0x7fb6280ae300 2023-12-12T09:53:18.259 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:18.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb60806ae40 msgr2=0x7fb60806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb60806ae40 0x7fb60806d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb618006fd0 tx=0x7fb618008040 comp rx=0 tx=0).stop 2023-12-12T09:53:18.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6280ae300 msgr2=0x7fb6280c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6280ae300 0x7fb6280c27c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb61000c450 tx=0x7fb61000c910 comp rx=0 tx=0).stop 2023-12-12T09:53:18.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 shutdown_connections 2023-12-12T09:53:18.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb60806ae40 0x7fb60806d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb62815a1e0 0x7fb6280ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6280ae300 0x7fb6280c27c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 --2- 172.21.15.72:0/967965393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb62806ab70 0x7fb62814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 >> 172.21.15.72:0/967965393 conn(0x7fb6280cee70 msgr2=0x7fb6280b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:18.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 shutdown_connections 2023-12-12T09:53:18.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.260+0000 7fb62ea1e640 1 -- 172.21.15.72:0/967965393 wait complete. 2023-12-12T09:53:18.281 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:18.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.496+0000 7fd8b8f41640 1 Processor -- start 2023-12-12T09:53:18.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.496+0000 7fd8b8f41640 1 -- start start 2023-12-12T09:53:18.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.497+0000 7fd8b8f41640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8b406d770 0x7fd8b40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.497+0000 7fd8b8f41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b4141960 0x7fd8b40cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.497+0000 7fd8b8f41640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b415b780 con 0x7fd8b406db50 2023-12-12T09:53:18.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.497+0000 7fd8b8f41640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b415b9d0 con 0x7fd8b406d770 2023-12-12T09:53:18.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.497+0000 7fd8b8f41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415bcb0 con 0x7fd8b4141960 2023-12-12T09:53:18.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.498+0000 7fd8b259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8b406d770 0x7fd8b40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.498+0000 7fd8b2d9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b4141960 0x7fd8b40cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.498+0000 7fd8b259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8b406d770 0x7fd8b40b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:54668/0 (socket says 172.21.15.72:54668) 2023-12-12T09:53:18.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.498+0000 7fd8b2d9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b4141960 0x7fd8b40cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:39214/0 (socket says 172.21.15.72:39214) 2023-12-12T09:53:18.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.498+0000 7fd8b259c640 1 -- 172.21.15.72:0/681555118 learned_addr learned my addr 172.21.15.72:0/681555118 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:18.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.499+0000 7fd8b1d9b640 1 -- 172.21.15.72:0/681555118 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276683000 0 0) 0x7fd8b415b780 con 0x7fd8b406db50 2023-12-12T09:53:18.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.499+0000 7fd8b1d9b640 1 -- 172.21.15.72:0/681555118 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a0003520 con 0x7fd8b406db50 2023-12-12T09:53:18.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.499+0000 7fd8b259c640 1 -- 172.21.15.72:0/681555118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b4141960 msgr2=0x7fd8b40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:18.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.499+0000 7fd8b259c640 1 --2- 172.21.15.72:0/681555118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b4141960 0x7fd8b40cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.499+0000 7fd8b259c640 1 -- 172.21.15.72:0/681555118 >> v1:172.21.15.113:6789/0 conn(0x7fd8b406db50 legacy=0x7fd8b4141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.499+0000 7fd8b259c640 1 -- 172.21.15.72:0/681555118 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b415bf90 con 0x7fd8b406d770 2023-12-12T09:53:18.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.499+0000 7fd8b2d9d640 1 --2- 172.21.15.72:0/681555118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b4141960 0x7fd8b40cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:53:18.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.500+0000 7fd8b259c640 1 --2- 172.21.15.72:0/681555118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406d770 0x7fd8b40b1e00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009870 tx=0x7fd8a402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d65547719b9aa4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.501+0000 7fd8b1d9b640 1 -- 172.21.15.72:0/681555118 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4030840 con 0x7fd8b406d770 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.501+0000 7fd8b1d9b640 1 -- 172.21.15.72:0/681555118 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a40309a0 con 0x7fd8b406d770 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.501+0000 7fd8b1d9b640 1 -- 172.21.15.72:0/681555118 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4030d30 con 0x7fd8b406d770 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.501+0000 7fd8b8f41640 1 -- 172.21.15.72:0/681555118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406d770 msgr2=0x7fd8b40b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.501+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/681555118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406d770 0x7fd8b40b1e00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009870 tx=0x7fd8a402b980 comp rx=0 tx=0).stop 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.502+0000 7fd8b8f41640 1 -- 172.21.15.72:0/681555118 shutdown_connections 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.502+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/681555118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b4141960 0x7fd8b40cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.502+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/681555118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406d770 0x7fd8b40b1e00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.502+0000 7fd8b8f41640 1 -- 172.21.15.72:0/681555118 >> 172.21.15.72:0/681555118 conn(0x7fd8b40cb850 msgr2=0x7fd8b40cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.502+0000 7fd8b8f41640 1 -- 172.21.15.72:0/681555118 shutdown_connections 2023-12-12T09:53:18.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.502+0000 7fd8b8f41640 1 -- 172.21.15.72:0/681555118 wait complete. 2023-12-12T09:53:18.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.503+0000 7fd8b8f41640 1 Processor -- start 2023-12-12T09:53:18.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.504+0000 7fd8b8f41640 1 -- start start 2023-12-12T09:53:18.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.504+0000 7fd8b8f41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b406d770 0x7fd8b415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.504+0000 7fd8b8f41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406db50 0x7fd8b40c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b8f41640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b4141960 0x7fd8b40c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b8f41640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415f350 con 0x7fd8b4141960 2023-12-12T09:53:18.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b8f41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415ac40 con 0x7fd8b406db50 2023-12-12T09:53:18.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b8f41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415e740 con 0x7fd8b406d770 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b2d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b4141960 0x7fd8b40c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b2d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b4141960 0x7fd8b40c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41848/0 (socket says 172.21.15.72:41848) 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b2d9d640 1 -- 172.21.15.72:0/71538345 learned_addr learned my addr 172.21.15.72:0/71538345 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b259c640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b406d770 0x7fd8b415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8a9d9b640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406db50 0x7fd8b40c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b2d9d640 1 -- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b406d770 msgr2=0x7fd8b415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b2d9d640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b406d770 0x7fd8b415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b2d9d640 1 -- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406db50 msgr2=0x7fd8b40c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b2d9d640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406db50 0x7fd8b40c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b2d9d640 1 -- 172.21.15.72:0/71538345 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac009a70 con 0x7fd8b4141960 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.505+0000 7fd8b259c640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b406d770 0x7fd8b415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.506+0000 7fd8b2d9d640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b4141960 0x7fd8b40c6440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00efc0 tx=0x7fd8ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.506+0000 7fd8ab7fe640 1 -- 172.21.15.72:0/71538345 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac009cd0 con 0x7fd8b4141960 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.506+0000 7fd8ab7fe640 1 -- 172.21.15.72:0/71538345 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac00f040 con 0x7fd8b4141960 2023-12-12T09:53:18.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.506+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a40094d0 con 0x7fd8b4141960 2023-12-12T09:53:18.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.506+0000 7fd8ab7fe640 1 -- 172.21.15.72:0/71538345 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac003990 con 0x7fd8b4141960 2023-12-12T09:53:18.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.506+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b40b3940 con 0x7fd8b4141960 2023-12-12T09:53:18.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.507+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b40745d0 con 0x7fd8b4141960 2023-12-12T09:53:18.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.508+0000 7fd8ab7fe640 1 -- 172.21.15.72:0/71538345 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8ac014480 con 0x7fd8b4141960 2023-12-12T09:53:18.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.509+0000 7fd8ab7fe640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd89406ae40 0x7fd89406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.509+0000 7fd8ab7fe640 1 -- 172.21.15.72:0/71538345 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd8ac08a6e0 con 0x7fd8b4141960 2023-12-12T09:53:18.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.509+0000 7fd8b259c640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd89406ae40 0x7fd89406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.510+0000 7fd8b259c640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd89406ae40 0x7fd89406d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4036000 tx=0x7fd8a403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.514+0000 7fd8ab7fe640 1 -- 172.21.15.72:0/71538345 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8ac0552a0 con 0x7fd8b4141960 2023-12-12T09:53:18.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.703+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8b4072770 con 0x7fd8b4141960 2023-12-12T09:53:18.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.704+0000 7fd8ab7fe640 1 -- 172.21.15.72:0/71538345 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8ac0588c0 con 0x7fd8b4141960 2023-12-12T09:53:18.706 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:18.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.706+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd89406ae40 msgr2=0x7fd89406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.706+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd89406ae40 0x7fd89406d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4036000 tx=0x7fd8a403b040 comp rx=0 tx=0).stop 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.706+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b4141960 msgr2=0x7fd8b40c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.706+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b4141960 0x7fd8b40c6440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00efc0 tx=0x7fd8ac00c490 comp rx=0 tx=0).stop 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.707+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 shutdown_connections 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.707+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd89406ae40 0x7fd89406d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.707+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b4141960 0x7fd8b40c6440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.707+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8b406db50 0x7fd8b40c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.707+0000 7fd8b8f41640 1 --2- 172.21.15.72:0/71538345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8b406d770 0x7fd8b415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.707+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 >> 172.21.15.72:0/71538345 conn(0x7fd8b40cb850 msgr2=0x7fd8b40acd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.707+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 shutdown_connections 2023-12-12T09:53:18.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.707+0000 7fd8b8f41640 1 -- 172.21.15.72:0/71538345 wait complete. 2023-12-12T09:53:18.728 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:18.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.948+0000 7fb03b553640 1 Processor -- start 2023-12-12T09:53:18.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.948+0000 7fb03b553640 1 -- start start 2023-12-12T09:53:18.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb03b553640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb03415af50 0x7fb03406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb03b553640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0340ce7e0 0x7fb03415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb03b553640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb034071cd0 con 0x7fb03406a460 2023-12-12T09:53:18.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb03b553640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0340717e0 con 0x7fb03415af50 2023-12-12T09:53:18.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb03b553640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03406c6b0 con 0x7fb0340ce7e0 2023-12-12T09:53:18.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb0392ef640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb03415af50 0x7fb03406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb0392ef640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb03415af50 0x7fb03406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41858/0 (socket says 172.21.15.72:41858) 2023-12-12T09:53:18.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb0392ef640 1 -- 172.21.15.72:0/2252222436 learned_addr learned my addr 172.21.15.72:0/2252222436 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:18.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb0392ef640 1 -- 172.21.15.72:0/2252222436 >> v2:172.21.15.113:3300/0 conn(0x7fb0340ce7e0 msgr2=0x7fb03415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:18.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb0392ef640 1 --2- 172.21.15.72:0/2252222436 >> v2:172.21.15.113:3300/0 conn(0x7fb0340ce7e0 0x7fb03415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.950+0000 7fb0392ef640 1 -- 172.21.15.72:0/2252222436 >> v1:172.21.15.113:6789/0 conn(0x7fb03406a460 legacy=0x7fb0340ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:18.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.951+0000 7fb0392ef640 1 -- 172.21.15.72:0/2252222436 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0340719f0 con 0x7fb03415af50 2023-12-12T09:53:18.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.952+0000 7fb0392ef640 1 --2- 172.21.15.72:0/2252222436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb03415af50 0x7fb03406c170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb03409d070 tx=0x7fb03002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deea3caa4ce552a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.952+0000 7fb02bfff640 1 -- 172.21.15.72:0/2252222436 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030031750 con 0x7fb03415af50 2023-12-12T09:53:18.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.952+0000 7fb02bfff640 1 -- 172.21.15.72:0/2252222436 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0300318b0 con 0x7fb03415af50 2023-12-12T09:53:18.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.952+0000 7fb02bfff640 1 -- 172.21.15.72:0/2252222436 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030031c40 con 0x7fb03415af50 2023-12-12T09:53:18.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.953+0000 7fb03b553640 1 -- 172.21.15.72:0/2252222436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb03415af50 msgr2=0x7fb03406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.953+0000 7fb03b553640 1 --2- 172.21.15.72:0/2252222436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb03415af50 0x7fb03406c170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb03409d070 tx=0x7fb03002b7f0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.953+0000 7fb03b553640 1 -- 172.21.15.72:0/2252222436 shutdown_connections 2023-12-12T09:53:18.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.953+0000 7fb03b553640 1 --2- 172.21.15.72:0/2252222436 >> v2:172.21.15.113:3300/0 conn(0x7fb0340ce7e0 0x7fb03415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.953+0000 7fb03b553640 1 --2- 172.21.15.72:0/2252222436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb03415af50 0x7fb03406c170 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.953+0000 7fb03b553640 1 -- 172.21.15.72:0/2252222436 >> 172.21.15.72:0/2252222436 conn(0x7fb0340cee70 msgr2=0x7fb0340cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:18.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.953+0000 7fb03b553640 1 -- 172.21.15.72:0/2252222436 shutdown_connections 2023-12-12T09:53:18.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.953+0000 7fb03b553640 1 -- 172.21.15.72:0/2252222436 wait complete. 2023-12-12T09:53:18.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.955+0000 7fb03b553640 1 Processor -- start 2023-12-12T09:53:18.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.955+0000 7fb03b553640 1 -- start start 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb03b553640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03406a460 0x7fb0340c1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb03b553640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0340ce7e0 0x7fb0340c2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb03b553640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03415af50 0x7fb0340ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb03b553640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340cacc0 con 0x7fb0340ce7e0 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb03b553640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034155230 con 0x7fb03406a460 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb03b553640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340c2ca0 con 0x7fb03415af50 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb038aee640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0340ce7e0 0x7fb0340c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb038aee640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0340ce7e0 0x7fb0340c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41860/0 (socket says 172.21.15.72:41860) 2023-12-12T09:53:18.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb038aee640 1 -- 172.21.15.72:0/208314642 learned_addr learned my addr 172.21.15.72:0/208314642 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb038aee640 1 -- 172.21.15.72:0/208314642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03415af50 msgr2=0x7fb0340ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb038aee640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03415af50 0x7fb0340ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb038aee640 1 -- 172.21.15.72:0/208314642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03406a460 msgr2=0x7fb0340c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb038aee640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03406a460 0x7fb0340c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.956+0000 7fb038aee640 1 -- 172.21.15.72:0/208314642 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0300094d0 con 0x7fb0340ce7e0 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.957+0000 7fb038aee640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0340ce7e0 0x7fb0340c2760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b4a0 tx=0x7fb02400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.957+0000 7fb029ffb640 1 -- 172.21.15.72:0/208314642 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024010610 con 0x7fb0340ce7e0 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.957+0000 7fb029ffb640 1 -- 172.21.15.72:0/208314642 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb024010770 con 0x7fb0340ce7e0 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.957+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034154900 con 0x7fb0340ce7e0 2023-12-12T09:53:18.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.957+0000 7fb029ffb640 1 -- 172.21.15.72:0/208314642 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024010b20 con 0x7fb0340ce7e0 2023-12-12T09:53:18.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.957+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb034154ec0 con 0x7fb0340ce7e0 2023-12-12T09:53:18.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.958+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb034074450 con 0x7fb0340ce7e0 2023-12-12T09:53:18.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.959+0000 7fb029ffb640 1 -- 172.21.15.72:0/208314642 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb024010c80 con 0x7fb0340ce7e0 2023-12-12T09:53:18.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.960+0000 7fb029ffb640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb01806ae40 0x7fb01806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:18.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.960+0000 7fb029ffb640 1 -- 172.21.15.72:0/208314642 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb024089640 con 0x7fb0340ce7e0 2023-12-12T09:53:18.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.960+0000 7fb0392ef640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb01806ae40 0x7fb01806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:18.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.961+0000 7fb0392ef640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb01806ae40 0x7fb01806d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb030035000 tx=0x7fb030030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:18.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:18.965+0000 7fb029ffb640 1 -- 172.21.15.72:0/208314642 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb024054320 con 0x7fb0340ce7e0 2023-12-12T09:53:19.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.161+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0341553a0 con 0x7fb0340ce7e0 2023-12-12T09:53:19.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.162+0000 7fb029ffb640 1 -- 172.21.15.72:0/208314642 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb024057940 con 0x7fb0340ce7e0 2023-12-12T09:53:19.164 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.164+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb01806ae40 msgr2=0x7fb01806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.164+0000 7fb03b553640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb01806ae40 0x7fb01806d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb030035000 tx=0x7fb030030000 comp rx=0 tx=0).stop 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.164+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0340ce7e0 msgr2=0x7fb0340c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.164+0000 7fb03b553640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0340ce7e0 0x7fb0340c2760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b4a0 tx=0x7fb02400b960 comp rx=0 tx=0).stop 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.165+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 shutdown_connections 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.165+0000 7fb03b553640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb01806ae40 0x7fb01806d300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.165+0000 7fb03b553640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03415af50 0x7fb0340ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.165+0000 7fb03b553640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0340ce7e0 0x7fb0340c2760 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.165+0000 7fb03b553640 1 --2- 172.21.15.72:0/208314642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03406a460 0x7fb0340c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.165+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 >> 172.21.15.72:0/208314642 conn(0x7fb0340cee70 msgr2=0x7fb034158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:19.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.165+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 shutdown_connections 2023-12-12T09:53:19.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.165+0000 7fb03b553640 1 -- 172.21.15.72:0/208314642 wait complete. 2023-12-12T09:53:19.186 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:19.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.395+0000 7ff5252c3640 1 Processor -- start 2023-12-12T09:53:19.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.396+0000 7ff5252c3640 1 -- start start 2023-12-12T09:53:19.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.397+0000 7ff5252c3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff52015d360 0x7ff5200d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.397+0000 7ff5252c3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5200cece0 0x7ff5200d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.397+0000 7ff5252c3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 0x7ff520160f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.397+0000 7ff5252c3640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200d1f00 con 0x7ff52015d360 2023-12-12T09:53:19.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.397+0000 7ff5252c3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200cdf20 con 0x7ff5200cece0 2023-12-12T09:53:19.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.397+0000 7ff5252c3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200ce200 con 0x7ff5200d1a70 2023-12-12T09:53:19.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.397+0000 7ff51ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff52015d360 0x7ff5200d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff51ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff52015d360 0x7ff5200d3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41864/0 (socket says 172.21.15.72:41864) 2023-12-12T09:53:19.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff51ffff640 1 -- 172.21.15.72:0/1721152793 learned_addr learned my addr 172.21.15.72:0/1721152793 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:19.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff524ac2640 1 --2- 172.21.15.72:0/1721152793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 0x7ff520160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff51f7fe640 1 --2- 172.21.15.72:0/1721152793 >> v2:172.21.15.113:3300/0 conn(0x7ff5200cece0 0x7ff5200d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff51ffff640 1 -- 172.21.15.72:0/1721152793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 msgr2=0x7ff520160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:19.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff51ffff640 1 --2- 172.21.15.72:0/1721152793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 0x7ff520160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff51ffff640 1 -- 172.21.15.72:0/1721152793 >> v2:172.21.15.113:3300/0 conn(0x7ff5200cece0 msgr2=0x7ff5200d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:19.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff51ffff640 1 --2- 172.21.15.72:0/1721152793 >> v2:172.21.15.113:3300/0 conn(0x7ff5200cece0 0x7ff5200d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.398+0000 7ff51ffff640 1 -- 172.21.15.72:0/1721152793 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200ce4e0 con 0x7ff52015d360 2023-12-12T09:53:19.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.399+0000 7ff51ffff640 1 --2- 172.21.15.72:0/1721152793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200d3660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff52008baa0 tx=0x7ff50802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b43efa0a5451259 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:19.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.399+0000 7ff51effd640 1 -- 172.21.15.72:0/1721152793 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508030860 con 0x7ff52015d360 2023-12-12T09:53:19.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff51effd640 1 -- 172.21.15.72:0/1721152793 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5080309c0 con 0x7ff52015d360 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff51effd640 1 -- 172.21.15.72:0/1721152793 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508030d50 con 0x7ff52015d360 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 -- 172.21.15.72:0/1721152793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 msgr2=0x7ff5200d3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 --2- 172.21.15.72:0/1721152793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200d3660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff52008baa0 tx=0x7ff50802b9a0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 -- 172.21.15.72:0/1721152793 shutdown_connections 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 --2- 172.21.15.72:0/1721152793 >> v2:172.21.15.113:3300/0 conn(0x7ff5200cece0 0x7ff5200d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 --2- 172.21.15.72:0/1721152793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 0x7ff520160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 --2- 172.21.15.72:0/1721152793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200d3660 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 -- 172.21.15.72:0/1721152793 >> 172.21.15.72:0/1721152793 conn(0x7ff52009c230 msgr2=0x7ff5200d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 -- 172.21.15.72:0/1721152793 shutdown_connections 2023-12-12T09:53:19.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.400+0000 7ff5252c3640 1 -- 172.21.15.72:0/1721152793 wait complete. 2023-12-12T09:53:19.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.402+0000 7ff5252c3640 1 Processor -- start 2023-12-12T09:53:19.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.402+0000 7ff5252c3640 1 -- start start 2023-12-12T09:53:19.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff5252c3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5200cece0 0x7ff5200b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff5252c3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 0x7ff5200b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff5252c3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff5252c3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200cad30 con 0x7ff52015d360 2023-12-12T09:53:19.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff5252c3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200caf40 con 0x7ff5200cece0 2023-12-12T09:53:19.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff5252c3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200cb220 con 0x7ff5200d1a70 2023-12-12T09:53:19.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff524ac2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff524ac2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41880/0 (socket says 172.21.15.72:41880) 2023-12-12T09:53:19.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.403+0000 7ff524ac2640 1 -- 172.21.15.72:0/2556441182 learned_addr learned my addr 172.21.15.72:0/2556441182 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:19.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff51f7fe640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 0x7ff5200b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff51ffff640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5200cece0 0x7ff5200b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff524ac2640 1 -- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 msgr2=0x7ff5200b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff524ac2640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 0x7ff5200b81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff524ac2640 1 -- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5200cece0 msgr2=0x7ff5200b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff524ac2640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5200cece0 0x7ff5200b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff524ac2640 1 -- 172.21.15.72:0/2556441182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080094d0 con 0x7ff52015d360 2023-12-12T09:53:19.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff51ffff640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5200cece0 0x7ff5200b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:19.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff524ac2640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200cbfe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff51800d490 tx=0x7ff51800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:19.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff51cff9640 1 -- 172.21.15.72:0/2556441182 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5180125f0 con 0x7ff52015d360 2023-12-12T09:53:19.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff51cff9640 1 -- 172.21.15.72:0/2556441182 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff518012770 con 0x7ff52015d360 2023-12-12T09:53:19.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.404+0000 7ff51cff9640 1 -- 172.21.15.72:0/2556441182 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518012b20 con 0x7ff52015d360 2023-12-12T09:53:19.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.405+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5200cb500 con 0x7ff52015d360 2023-12-12T09:53:19.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.405+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5200cbac0 con 0x7ff52015d360 2023-12-12T09:53:19.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.406+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5200678e0 con 0x7ff52015d360 2023-12-12T09:53:19.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.407+0000 7ff51cff9640 1 -- 172.21.15.72:0/2556441182 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff518012c80 con 0x7ff52015d360 2023-12-12T09:53:19.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.408+0000 7ff51cff9640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff50006ae40 0x7ff50006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.408+0000 7ff51cff9640 1 -- 172.21.15.72:0/2556441182 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff518089e90 con 0x7ff52015d360 2023-12-12T09:53:19.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.408+0000 7ff51ffff640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff50006ae40 0x7ff50006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.408+0000 7ff51ffff640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff50006ae40 0x7ff50006d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff508036000 tx=0x7ff50803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:19.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.412+0000 7ff51cff9640 1 -- 172.21.15.72:0/2556441182 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff518054ae0 con 0x7ff52015d360 2023-12-12T09:53:19.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.602+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff5200d21f0 con 0x7ff52015d360 2023-12-12T09:53:19.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.602+0000 7ff51cff9640 1 -- 172.21.15.72:0/2556441182 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff518058100 con 0x7ff52015d360 2023-12-12T09:53:19.604 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff50006ae40 msgr2=0x7ff50006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff50006ae40 0x7ff50006d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff508036000 tx=0x7ff50803b040 comp rx=0 tx=0).stop 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 msgr2=0x7ff5200cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200cbfe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff51800d490 tx=0x7ff51800d950 comp rx=0 tx=0).stop 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 shutdown_connections 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff50006ae40 0x7ff50006d300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff52015d360 0x7ff5200cbfe0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5200d1a70 0x7ff5200b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 --2- 172.21.15.72:0/2556441182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5200cece0 0x7ff5200b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 >> 172.21.15.72:0/2556441182 conn(0x7ff52009c230 msgr2=0x7ff5200b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 shutdown_connections 2023-12-12T09:53:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.605+0000 7ff5252c3640 1 -- 172.21.15.72:0/2556441182 wait complete. 2023-12-12T09:53:19.626 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:53:19.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.835+0000 7ffa6a75a640 1 Processor -- start 2023-12-12T09:53:19.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.835+0000 7ffa6a75a640 1 -- start start 2023-12-12T09:53:19.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.837+0000 7ffa6a75a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ffa6415d580 0x7ffa640adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.837+0000 7ffa6a75a640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa640b07f0 con 0x7ffa6406c690 2023-12-12T09:53:19.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.837+0000 7ffa6a75a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa640b0a40 con 0x7ffa640cfc50 2023-12-12T09:53:19.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.837+0000 7ffa6a75a640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6415d960 con 0x7ffa6415d580 2023-12-12T09:53:19.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.837+0000 7ffa63fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ffa6415d580 0x7ffa640adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.837+0000 7ffa68cf7640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ffa6406c690 0x7ffa640b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42144/0 (socket says 172.21.15.72:42144) 2023-12-12T09:53:19.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.837+0000 7ffa68cf7640 1 -- 172.21.15.72:0/3401886416 learned_addr learned my addr 172.21.15.72:0/3401886416 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:19.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.838+0000 7ffa63fff640 1 -- 172.21.15.72:0/3401886416 >> v1:172.21.15.113:6789/0 conn(0x7ffa640cfc50 legacy=0x7ffa6406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:19.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.838+0000 7ffa63fff640 1 -- 172.21.15.72:0/3401886416 >> v1:172.21.15.72:6789/0 conn(0x7ffa6406c690 legacy=0x7ffa640b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.838+0000 7ffa63fff640 1 -- 172.21.15.72:0/3401886416 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa640b0d20 con 0x7ffa6415d580 2023-12-12T09:53:19.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.839+0000 7ffa63fff640 1 --2- 172.21.15.72:0/3401886416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6415d580 0x7ffa640adc60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009d90 tx=0x7ffa5402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4ba1e14bb3e172 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:19.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.839+0000 7ffa62ffd640 1 -- 172.21.15.72:0/3401886416 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5402cc60 con 0x7ffa6415d580 2023-12-12T09:53:19.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.839+0000 7ffa62ffd640 1 -- 172.21.15.72:0/3401886416 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5402cdc0 con 0x7ffa6415d580 2023-12-12T09:53:19.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.839+0000 7ffa62ffd640 1 -- 172.21.15.72:0/3401886416 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54034660 con 0x7ffa6415d580 2023-12-12T09:53:19.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.840+0000 7ffa6a75a640 1 -- 172.21.15.72:0/3401886416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6415d580 msgr2=0x7ffa640adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.840+0000 7ffa6a75a640 1 --2- 172.21.15.72:0/3401886416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6415d580 0x7ffa640adc60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009d90 tx=0x7ffa5402beb0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.840+0000 7ffa6a75a640 1 -- 172.21.15.72:0/3401886416 shutdown_connections 2023-12-12T09:53:19.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.840+0000 7ffa6a75a640 1 --2- 172.21.15.72:0/3401886416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6415d580 0x7ffa640adc60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.840+0000 7ffa6a75a640 1 -- 172.21.15.72:0/3401886416 >> 172.21.15.72:0/3401886416 conn(0x7ffa640d0e90 msgr2=0x7ffa640d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:19.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.840+0000 7ffa6a75a640 1 -- 172.21.15.72:0/3401886416 shutdown_connections 2023-12-12T09:53:19.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.840+0000 7ffa6a75a640 1 -- 172.21.15.72:0/3401886416 wait complete. 2023-12-12T09:53:19.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.841+0000 7ffa6a75a640 1 Processor -- start 2023-12-12T09:53:19.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.841+0000 7ffa6a75a640 1 -- start start 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.842+0000 7ffa6a75a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6406c690 0x7ffa640b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.842+0000 7ffa6a75a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa640cfc50 0x7ffa6414f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.842+0000 7ffa6a75a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa6415d580 0x7ffa6409af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.842+0000 7ffa6a75a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640c73f0 con 0x7ffa6406c690 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.842+0000 7ffa6a75a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6409cf60 con 0x7ffa640cfc50 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.842+0000 7ffa6a75a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640c7110 con 0x7ffa6415d580 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6406c690 0x7ffa640b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6406c690 0x7ffa640b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41886/0 (socket says 172.21.15.72:41886) 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 -- 172.21.15.72:0/2455324543 learned_addr learned my addr 172.21.15.72:0/2455324543 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 -- 172.21.15.72:0/2455324543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa6415d580 msgr2=0x7ffa6409af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa6415d580 0x7ffa6409af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 -- 172.21.15.72:0/2455324543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa640cfc50 msgr2=0x7ffa6414f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:19.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa640cfc50 0x7ffa6414f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:19.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 -- 172.21.15.72:0/2455324543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54009a70 con 0x7ffa6406c690 2023-12-12T09:53:19.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa63fff640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6406c690 0x7ffa640b47f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa54037040 tx=0x7ffa54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:19.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.843+0000 7ffa60ff9640 1 -- 172.21.15.72:0/2455324543 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54040650 con 0x7ffa6406c690 2023-12-12T09:53:19.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.844+0000 7ffa60ff9640 1 -- 172.21.15.72:0/2455324543 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa540407b0 con 0x7ffa6406c690 2023-12-12T09:53:19.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.844+0000 7ffa60ff9640 1 -- 172.21.15.72:0/2455324543 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54040b60 con 0x7ffa6406c690 2023-12-12T09:53:19.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.844+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6409d200 con 0x7ffa6406c690 2023-12-12T09:53:19.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.844+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6409d7c0 con 0x7ffa6406c690 2023-12-12T09:53:19.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.845+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa640d00c0 con 0x7ffa6406c690 2023-12-12T09:53:19.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.846+0000 7ffa60ff9640 1 -- 172.21.15.72:0/2455324543 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa54040cc0 con 0x7ffa6406c690 2023-12-12T09:53:19.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.846+0000 7ffa60ff9640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa4406ae40 0x7ffa4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:19.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.847+0000 7ffa60ff9640 1 -- 172.21.15.72:0/2455324543 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffa540ac940 con 0x7ffa6406c690 2023-12-12T09:53:19.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.847+0000 7ffa637fe640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa4406ae40 0x7ffa4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:19.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.847+0000 7ffa637fe640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa4406ae40 0x7ffa4406d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffa6409edf0 tx=0x7ffa4c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:19.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:19.851+0000 7ffa60ff9640 1 -- 172.21.15.72:0/2455324543 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa640d00c0 con 0x7ffa6406c690 2023-12-12T09:53:20.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.044+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffa640d00c0 con 0x7ffa6406c690 2023-12-12T09:53:20.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.045+0000 7ffa60ff9640 1 -- 172.21.15.72:0/2455324543 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffa640d00c0 con 0x7ffa6406c690 2023-12-12T09:53:20.046 INFO:teuthology.orchestra.run.smithi072.stderr:listed 0 entries 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa4406ae40 msgr2=0x7ffa4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa4406ae40 0x7ffa4406d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffa6409edf0 tx=0x7ffa4c0079b0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6406c690 msgr2=0x7ffa640b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6406c690 0x7ffa640b47f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa54037040 tx=0x7ffa54004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 shutdown_connections 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa4406ae40 0x7ffa4406d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa6415d580 0x7ffa6409af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa640cfc50 0x7ffa6414f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 --2- 172.21.15.72:0/2455324543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa6406c690 0x7ffa640b47f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 >> 172.21.15.72:0/2455324543 conn(0x7ffa640d0e90 msgr2=0x7ffa640d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 shutdown_connections 2023-12-12T09:53:20.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.047+0000 7ffa6a75a640 1 -- 172.21.15.72:0/2455324543 wait complete. 2023-12-12T09:53:20.065 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-12T09:53:20.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:20.066 DEBUG:teuthology.orchestra.run.smithi072:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-12-12T09:53:20.157 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:53:20.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:20.157 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-12T09:53:20.210 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:53:20.211 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:20.271 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:53:20.271 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:20.327 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:53:20.327 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:20.381 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:20.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.605+0000 7fdc97fff640 1 Processor -- start 2023-12-12T09:53:20.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.606+0000 7fdc97fff640 1 -- start start 2023-12-12T09:53:20.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.607+0000 7fdc97fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc980ae200 0x7fdc98070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:20.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.607+0000 7fdc97fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdc980abd20 0x7fdc98071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:20.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.607+0000 7fdc97fff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc98071f90 con 0x7fdc9815e5d0 2023-12-12T09:53:20.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.607+0000 7fdc97fff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc98072270 con 0x7fdc980abd20 2023-12-12T09:53:20.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.607+0000 7fdc97fff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc98070070 con 0x7fdc980ae200 2023-12-12T09:53:20.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.607+0000 7fdc977fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdc980abd20 0x7fdc98071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:20.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.608+0000 7fdc977fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdc980abd20 0x7fdc98071090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41898/0 (socket says 172.21.15.72:41898) 2023-12-12T09:53:20.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.608+0000 7fdc977fe640 1 -- 172.21.15.72:0/1301223047 learned_addr learned my addr 172.21.15.72:0/1301223047 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:20.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.608+0000 7fdc8e7fc640 1 --2- 172.21.15.72:0/1301223047 >> v2:172.21.15.113:3300/0 conn(0x7fdc980ae200 0x7fdc98070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:20.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.608+0000 7fdc977fe640 1 -- 172.21.15.72:0/1301223047 >> v2:172.21.15.113:3300/0 conn(0x7fdc980ae200 msgr2=0x7fdc98070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:20.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.608+0000 7fdc977fe640 1 --2- 172.21.15.72:0/1301223047 >> v2:172.21.15.113:3300/0 conn(0x7fdc980ae200 0x7fdc98070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.608+0000 7fdc977fe640 1 -- 172.21.15.72:0/1301223047 >> v1:172.21.15.113:6789/0 conn(0x7fdc9815e5d0 legacy=0x7fdc9806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:20.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.608+0000 7fdc977fe640 1 -- 172.21.15.72:0/1301223047 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98070350 con 0x7fdc980abd20 2023-12-12T09:53:20.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.608+0000 7fdc8e7fc640 1 --2- 172.21.15.72:0/1301223047 >> v2:172.21.15.113:3300/0 conn(0x7fdc980ae200 0x7fdc98070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:20.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.609+0000 7fdc977fe640 1 --2- 172.21.15.72:0/1301223047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98071090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdc980739e0 tx=0x7fdc9002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45eb6a27ad8a844 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:20.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.609+0000 7fdc967fc640 1 -- 172.21.15.72:0/1301223047 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90031750 con 0x7fdc980abd20 2023-12-12T09:53:20.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.610+0000 7fdc967fc640 1 -- 172.21.15.72:0/1301223047 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc900318b0 con 0x7fdc980abd20 2023-12-12T09:53:20.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.610+0000 7fdc967fc640 1 -- 172.21.15.72:0/1301223047 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90031c40 con 0x7fdc980abd20 2023-12-12T09:53:20.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.610+0000 7fdc97fff640 1 -- 172.21.15.72:0/1301223047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 msgr2=0x7fdc98071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:20.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.610+0000 7fdc97fff640 1 --2- 172.21.15.72:0/1301223047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98071090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdc980739e0 tx=0x7fdc9002b7f0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.610+0000 7fdc97fff640 1 -- 172.21.15.72:0/1301223047 shutdown_connections 2023-12-12T09:53:20.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.610+0000 7fdc97fff640 1 --2- 172.21.15.72:0/1301223047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98071090 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.610+0000 7fdc97fff640 1 --2- 172.21.15.72:0/1301223047 >> v2:172.21.15.113:3300/0 conn(0x7fdc980ae200 0x7fdc98070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.610+0000 7fdc97fff640 1 -- 172.21.15.72:0/1301223047 >> 172.21.15.72:0/1301223047 conn(0x7fdc980b0860 msgr2=0x7fdc980b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:20.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.611+0000 7fdc97fff640 1 -- 172.21.15.72:0/1301223047 shutdown_connections 2023-12-12T09:53:20.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.611+0000 7fdc97fff640 1 -- 172.21.15.72:0/1301223047 wait complete. 2023-12-12T09:53:20.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.612+0000 7fdc97fff640 1 Processor -- start 2023-12-12T09:53:20.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.612+0000 7fdc97fff640 1 -- start start 2023-12-12T09:53:20.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.613+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.613+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc980ae200 0x7fdc981446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.613+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815e5d0 0x7fdc98144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.613+0000 7fdc97fff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980b2210 con 0x7fdc980abd20 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.613+0000 7fdc97fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980b24f0 con 0x7fdc980ae200 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.613+0000 7fdc97fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980b27d0 con 0x7fdc9815e5d0 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41908/0 (socket says 172.21.15.72:41908) 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 -- 172.21.15.72:0/305814421 learned_addr learned my addr 172.21.15.72:0/305814421 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 -- 172.21.15.72:0/305814421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815e5d0 msgr2=0x7fdc98144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815e5d0 0x7fdc98144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 -- 172.21.15.72:0/305814421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc980ae200 msgr2=0x7fdc981446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc980ae200 0x7fdc981446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 -- 172.21.15.72:0/305814421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc900094d0 con 0x7fdc980abd20 2023-12-12T09:53:20.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.614+0000 7fdc96ffd640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98154420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800ba50 tx=0x7fdc8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:20.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.615+0000 7fdc94ff9640 1 -- 172.21.15.72:0/305814421 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800cc10 con 0x7fdc980abd20 2023-12-12T09:53:20.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.615+0000 7fdc94ff9640 1 -- 172.21.15.72:0/305814421 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8800cd70 con 0x7fdc980abd20 2023-12-12T09:53:20.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.615+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9806c680 con 0x7fdc980abd20 2023-12-12T09:53:20.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.615+0000 7fdc94ff9640 1 -- 172.21.15.72:0/305814421 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88014680 con 0x7fdc980abd20 2023-12-12T09:53:20.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.615+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc980b2d90 con 0x7fdc980abd20 2023-12-12T09:53:20.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.616+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc980b2fb0 con 0x7fdc980abd20 2023-12-12T09:53:20.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.617+0000 7fdc94ff9640 1 -- 172.21.15.72:0/305814421 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc8801d3b0 con 0x7fdc980abd20 2023-12-12T09:53:20.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.618+0000 7fdc94ff9640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdc7806ae40 0x7fdc7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:20.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.618+0000 7fdc94ff9640 1 -- 172.21.15.72:0/305814421 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdc88089af0 con 0x7fdc980abd20 2023-12-12T09:53:20.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.618+0000 7fdc8e7fc640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdc7806ae40 0x7fdc7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:20.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.619+0000 7fdc8e7fc640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdc7806ae40 0x7fdc7806d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc80006fd0 tx=0x7fdc80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:20.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.622+0000 7fdc94ff9640 1 -- 172.21.15.72:0/305814421 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc88054740 con 0x7fdc980abd20 2023-12-12T09:53:20.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.812+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc98074450 con 0x7fdc980abd20 2023-12-12T09:53:20.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.815+0000 7fdc94ff9640 1 -- 172.21.15.72:0/305814421 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7fdc88019090 con 0x7fdc980abd20 2023-12-12T09:53:20.818 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:20.818 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdc7806ae40 msgr2=0x7fdc7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdc7806ae40 0x7fdc7806d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc80006fd0 tx=0x7fdc80008040 comp rx=0 tx=0).stop 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 msgr2=0x7fdc98154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98154420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800ba50 tx=0x7fdc8800bf10 comp rx=0 tx=0).stop 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 shutdown_connections 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdc7806ae40 0x7fdc7806d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815e5d0 0x7fdc98144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc980ae200 0x7fdc981446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 --2- 172.21.15.72:0/305814421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc980abd20 0x7fdc98154420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.818+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 >> 172.21.15.72:0/305814421 conn(0x7fdc980b0860 msgr2=0x7fdc9806bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.819+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 shutdown_connections 2023-12-12T09:53:20.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:20.819+0000 7fdc97fff640 1 -- 172.21.15.72:0/305814421 wait complete. 2023-12-12T09:53:20.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:20.842 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:21.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.056+0000 7fa9c1b61640 1 Processor -- start 2023-12-12T09:53:21.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.056+0000 7fa9c1b61640 1 -- start start 2023-12-12T09:53:21.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.057+0000 7fa9c1b61640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa9bc071fb0 0x7fa9bc0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.057+0000 7fa9c1b61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9bc0ca4f0 0x7fa9bc0c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.057+0000 7fa9c1b61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 0x7fa9bc0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.057+0000 7fa9c1b61640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc0ca960 con 0x7fa9bc071fb0 2023-12-12T09:53:21.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.057+0000 7fa9c1b61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc0c2630 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.057+0000 7fa9c1b61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0caf80 con 0x7fa9bc0c21c0 2023-12-12T09:53:21.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9bb7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa9bc071fb0 0x7fa9bc0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9bb7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa9bc071fb0 0x7fa9bc0ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41918/0 (socket says 172.21.15.72:41918) 2023-12-12T09:53:21.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9bb7fe640 1 -- 172.21.15.72:0/835121175 learned_addr learned my addr 172.21.15.72:0/835121175 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:21.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9bb7fe640 1 -- 172.21.15.72:0/835121175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 msgr2=0x7fa9bc0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:21.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9bb7fe640 1 --2- 172.21.15.72:0/835121175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 0x7fa9bc0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9bb7fe640 1 -- 172.21.15.72:0/835121175 >> v2:172.21.15.113:3300/0 conn(0x7fa9bc0ca4f0 msgr2=0x7fa9bc0c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:21.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9b3fff640 1 --2- 172.21.15.72:0/835121175 >> v2:172.21.15.113:3300/0 conn(0x7fa9bc0ca4f0 0x7fa9bc0c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9bb7fe640 1 --2- 172.21.15.72:0/835121175 >> v2:172.21.15.113:3300/0 conn(0x7fa9bc0ca4f0 0x7fa9bc0c1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.058+0000 7fa9bb7fe640 1 -- 172.21.15.72:0/835121175 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc06a5e0 con 0x7fa9bc071fb0 2023-12-12T09:53:21.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.059+0000 7fa9bb7fe640 1 --2- 172.21.15.72:0/835121175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc071fb0 0x7fa9bc0ce490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0ce9d0 tx=0x7fa9ac02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bdd39cc22bfd81 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.060+0000 7fa9baffd640 1 -- 172.21.15.72:0/835121175 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac031860 con 0x7fa9bc071fb0 2023-12-12T09:53:21.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.060+0000 7fa9baffd640 1 -- 172.21.15.72:0/835121175 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac0319c0 con 0x7fa9bc071fb0 2023-12-12T09:53:21.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.060+0000 7fa9baffd640 1 -- 172.21.15.72:0/835121175 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac031d50 con 0x7fa9bc071fb0 2023-12-12T09:53:21.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.060+0000 7fa9c1b61640 1 -- 172.21.15.72:0/835121175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc071fb0 msgr2=0x7fa9bc0ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.060+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/835121175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc071fb0 0x7fa9bc0ce490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0ce9d0 tx=0x7fa9ac02b900 comp rx=0 tx=0).stop 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.061+0000 7fa9c1b61640 1 -- 172.21.15.72:0/835121175 shutdown_connections 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.061+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/835121175 >> v2:172.21.15.113:3300/0 conn(0x7fa9bc0ca4f0 0x7fa9bc0c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.061+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/835121175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 0x7fa9bc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.061+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/835121175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc071fb0 0x7fa9bc0ce490 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.061+0000 7fa9c1b61640 1 -- 172.21.15.72:0/835121175 >> 172.21.15.72:0/835121175 conn(0x7fa9bc0cee70 msgr2=0x7fa9bc0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.061+0000 7fa9c1b61640 1 -- 172.21.15.72:0/835121175 shutdown_connections 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.061+0000 7fa9c1b61640 1 -- 172.21.15.72:0/835121175 wait complete. 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.063+0000 7fa9c1b61640 1 Processor -- start 2023-12-12T09:53:21.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.063+0000 7fa9c1b61640 1 -- start start 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.064+0000 7fa9c1b61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9bc071fb0 0x7fa9bc160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.064+0000 7fa9c1b61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 0x7fa9bc0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.064+0000 7fa9c1b61640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc0ca4f0 0x7fa9bc0adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.064+0000 7fa9c1b61640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0c5fe0 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.064+0000 7fa9c1b61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0ad780 con 0x7fa9bc071fb0 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.064+0000 7fa9c1b61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc1605a0 con 0x7fa9bc0c21c0 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc0ca4f0 0x7fa9bc0adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc0ca4f0 0x7fa9bc0adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41932/0 (socket says 172.21.15.72:41932) 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bbfff640 1 -- 172.21.15.72:0/2091944995 learned_addr learned my addr 172.21.15.72:0/2091944995 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:21.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bb7fe640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9bc071fb0 0x7fa9bc160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9b3fff640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 0x7fa9bc0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bbfff640 1 -- 172.21.15.72:0/2091944995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 msgr2=0x7fa9bc0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bbfff640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 0x7fa9bc0c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bbfff640 1 -- 172.21.15.72:0/2091944995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9bc071fb0 msgr2=0x7fa9bc160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bbfff640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9bc071fb0 0x7fa9bc160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.065+0000 7fa9bbfff640 1 -- 172.21.15.72:0/2091944995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0094d0 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.066+0000 7fa9bbfff640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc0ca4f0 0x7fa9bc0adeb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400b9a0 tx=0x7fa9b400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.066+0000 7fa9b8ff9640 1 -- 172.21.15.72:0/2091944995 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400cba0 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.066+0000 7fa9b8ff9640 1 -- 172.21.15.72:0/2091944995 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b400cd20 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.066+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc0acee0 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.066+0000 7fa9b8ff9640 1 -- 172.21.15.72:0/2091944995 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4014680 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.066+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc0ad4a0 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.067+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc0c6120 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.068+0000 7fa9b8ff9640 1 -- 172.21.15.72:0/2091944995 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9b401d3b0 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.069+0000 7fa9b8ff9640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa99c06ae40 0x7fa99c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.069+0000 7fa9b8ff9640 1 -- 172.21.15.72:0/2091944995 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa9b40898e0 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.069+0000 7fa9bb7fe640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa99c06ae40 0x7fa99c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.070+0000 7fa9bb7fe640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa99c06ae40 0x7fa99c06d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac036000 tx=0x7fa9ac03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.073+0000 7fa9b8ff9640 1 -- 172.21.15.72:0/2091944995 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9bc0c6120 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.267+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9bc0c6120 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.267+0000 7fa9b8ff9640 1 -- 172.21.15.72:0/2091944995 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9bc0c6120 con 0x7fa9bc0ca4f0 2023-12-12T09:53:21.269 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:21.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa99c06ae40 msgr2=0x7fa99c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa99c06ae40 0x7fa99c06d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac036000 tx=0x7fa9ac03c040 comp rx=0 tx=0).stop 2023-12-12T09:53:21.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc0ca4f0 msgr2=0x7fa9bc0adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc0ca4f0 0x7fa9bc0adeb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400b9a0 tx=0x7fa9b400be60 comp rx=0 tx=0).stop 2023-12-12T09:53:21.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 shutdown_connections 2023-12-12T09:53:21.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa99c06ae40 0x7fa99c06d300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9bc0ca4f0 0x7fa9bc0adeb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9bc0c21c0 0x7fa9bc0c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 --2- 172.21.15.72:0/2091944995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9bc071fb0 0x7fa9bc160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 >> 172.21.15.72:0/2091944995 conn(0x7fa9bc0cee70 msgr2=0x7fa9bc0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:21.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 shutdown_connections 2023-12-12T09:53:21.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.270+0000 7fa9c1b61640 1 -- 172.21.15.72:0/2091944995 wait complete. 2023-12-12T09:53:21.291 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:21.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.504+0000 7fcfde061640 1 Processor -- start 2023-12-12T09:53:21.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.505+0000 7fcfde061640 1 -- start start 2023-12-12T09:53:21.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.506+0000 7fcfde061640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fcfd81609b0 0x7fcfd80d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.506+0000 7fcfde061640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfd80d1220 0x7fcfd80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.506+0000 7fcfde061640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd80d04b0 con 0x7fcfd80d0060 2023-12-12T09:53:21.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.506+0000 7fcfde061640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd8160d90 con 0x7fcfd81609b0 2023-12-12T09:53:21.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.506+0000 7fcfde061640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd815dd60 con 0x7fcfd80d1220 2023-12-12T09:53:21.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.506+0000 7fcfd77fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fcfd81609b0 0x7fcfd80d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.506+0000 7fcfd77fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fcfd81609b0 0x7fcfd80d0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41942/0 (socket says 172.21.15.72:41942) 2023-12-12T09:53:21.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.506+0000 7fcfd77fe640 1 -- 172.21.15.72:0/527457756 learned_addr learned my addr 172.21.15.72:0/527457756 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:21.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.507+0000 7fcfd6ffd640 1 --2- 172.21.15.72:0/527457756 >> v2:172.21.15.113:3300/0 conn(0x7fcfd80d1220 0x7fcfd80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.507+0000 7fcfd77fe640 1 -- 172.21.15.72:0/527457756 >> v2:172.21.15.113:3300/0 conn(0x7fcfd80d1220 msgr2=0x7fcfd80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:21.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.507+0000 7fcfd77fe640 1 --2- 172.21.15.72:0/527457756 >> v2:172.21.15.113:3300/0 conn(0x7fcfd80d1220 0x7fcfd80cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.507+0000 7fcfd77fe640 1 -- 172.21.15.72:0/527457756 >> v1:172.21.15.72:6789/0 conn(0x7fcfd80d0060 legacy=0x7fcfd815cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.507+0000 7fcfd77fe640 1 -- 172.21.15.72:0/527457756 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd815e040 con 0x7fcfd81609b0 2023-12-12T09:53:21.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.508+0000 7fcfd77fe640 1 --2- 172.21.15.72:0/527457756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd81609b0 0x7fcfd80d0ce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcfd808d680 tx=0x7fcfc802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b87574d96d3609 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.508+0000 7fcfd67fc640 1 -- 172.21.15.72:0/527457756 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc802cb10 con 0x7fcfd81609b0 2023-12-12T09:53:21.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.508+0000 7fcfd67fc640 1 -- 172.21.15.72:0/527457756 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc802cc70 con 0x7fcfd81609b0 2023-12-12T09:53:21.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.508+0000 7fcfd67fc640 1 -- 172.21.15.72:0/527457756 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80343f0 con 0x7fcfd81609b0 2023-12-12T09:53:21.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.509+0000 7fcfde061640 1 -- 172.21.15.72:0/527457756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd81609b0 msgr2=0x7fcfd80d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.509+0000 7fcfde061640 1 --2- 172.21.15.72:0/527457756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd81609b0 0x7fcfd80d0ce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcfd808d680 tx=0x7fcfc802bf70 comp rx=0 tx=0).stop 2023-12-12T09:53:21.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.509+0000 7fcfde061640 1 -- 172.21.15.72:0/527457756 shutdown_connections 2023-12-12T09:53:21.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.509+0000 7fcfde061640 1 --2- 172.21.15.72:0/527457756 >> v2:172.21.15.113:3300/0 conn(0x7fcfd80d1220 0x7fcfd80cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.509+0000 7fcfde061640 1 --2- 172.21.15.72:0/527457756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd81609b0 0x7fcfd80d0ce0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.509+0000 7fcfde061640 1 -- 172.21.15.72:0/527457756 >> 172.21.15.72:0/527457756 conn(0x7fcfd809c450 msgr2=0x7fcfd80d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:21.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.509+0000 7fcfde061640 1 -- 172.21.15.72:0/527457756 shutdown_connections 2023-12-12T09:53:21.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.509+0000 7fcfde061640 1 -- 172.21.15.72:0/527457756 wait complete. 2023-12-12T09:53:21.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.511+0000 7fcfde061640 1 Processor -- start 2023-12-12T09:53:21.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.511+0000 7fcfde061640 1 -- start start 2023-12-12T09:53:21.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfde061640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd80d0060 0x7fcfd81f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfde061640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfd80d1220 0x7fcfd80c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfde061640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfd81609b0 0x7fcfd815eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfde061640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd814ee10 con 0x7fcfd80d0060 2023-12-12T09:53:21.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfde061640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81f8030 con 0x7fcfd81609b0 2023-12-12T09:53:21.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfde061640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80c9740 con 0x7fcfd80d1220 2023-12-12T09:53:21.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfd77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd80d0060 0x7fcfd81f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfd77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd80d0060 0x7fcfd81f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41948/0 (socket says 172.21.15.72:41948) 2023-12-12T09:53:21.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.512+0000 7fcfd77fe640 1 -- 172.21.15.72:0/2384986748 learned_addr learned my addr 172.21.15.72:0/2384986748 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.513+0000 7fcfd77fe640 1 -- 172.21.15.72:0/2384986748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfd80d1220 msgr2=0x7fcfd80c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.513+0000 7fcfd77fe640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfd80d1220 0x7fcfd80c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.513+0000 7fcfd77fe640 1 -- 172.21.15.72:0/2384986748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfd81609b0 msgr2=0x7fcfd815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.513+0000 7fcfd7fff640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfd81609b0 0x7fcfd815eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.513+0000 7fcfd77fe640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfd81609b0 0x7fcfd815eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.513+0000 7fcfd77fe640 1 -- 172.21.15.72:0/2384986748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8009b30 con 0x7fcfd80d0060 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.513+0000 7fcfd77fe640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd80d0060 0x7fcfd81f35e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8036040 tx=0x7fcfc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.514+0000 7fcfcffff640 1 -- 172.21.15.72:0/2384986748 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8009510 con 0x7fcfd80d0060 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.514+0000 7fcfcffff640 1 -- 172.21.15.72:0/2384986748 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc803e450 con 0x7fcfd80d0060 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.514+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd81f74b0 con 0x7fcfd80d0060 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.514+0000 7fcfcffff640 1 -- 172.21.15.72:0/2384986748 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc803e800 con 0x7fcfd80d0060 2023-12-12T09:53:21.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.514+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd81f7a70 con 0x7fcfd80d0060 2023-12-12T09:53:21.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.515+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd80745d0 con 0x7fcfd80d0060 2023-12-12T09:53:21.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.516+0000 7fcfcffff640 1 -- 172.21.15.72:0/2384986748 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcfc803e960 con 0x7fcfd80d0060 2023-12-12T09:53:21.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.517+0000 7fcfcffff640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfb806ae70 0x7fcfb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.517+0000 7fcfcffff640 1 -- 172.21.15.72:0/2384986748 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcfc80ac790 con 0x7fcfd80d0060 2023-12-12T09:53:21.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.517+0000 7fcfd6ffd640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfb806ae70 0x7fcfb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.518+0000 7fcfd6ffd640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfb806ae70 0x7fcfb806d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006fd0 tx=0x7fcfc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.521+0000 7fcfcffff640 1 -- 172.21.15.72:0/2384986748 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfc8077470 con 0x7fcfd80d0060 2023-12-12T09:53:21.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.712+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfd80c69e0 con 0x7fcfd80d0060 2023-12-12T09:53:21.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.713+0000 7fcfcffff640 1 -- 172.21.15.72:0/2384986748 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfc807aa90 con 0x7fcfd80d0060 2023-12-12T09:53:21.715 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:21.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfb806ae70 msgr2=0x7fcfb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfb806ae70 0x7fcfb806d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006fd0 tx=0x7fcfc0008040 comp rx=0 tx=0).stop 2023-12-12T09:53:21.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd80d0060 msgr2=0x7fcfd81f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd80d0060 0x7fcfd81f35e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8036040 tx=0x7fcfc8004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:21.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 shutdown_connections 2023-12-12T09:53:21.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfb806ae70 0x7fcfb806d330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfd81609b0 0x7fcfd815eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfd80d1220 0x7fcfd80c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 --2- 172.21.15.72:0/2384986748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfd80d0060 0x7fcfd81f35e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 >> 172.21.15.72:0/2384986748 conn(0x7fcfd809c450 msgr2=0x7fcfd80d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:21.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 shutdown_connections 2023-12-12T09:53:21.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.716+0000 7fcfde061640 1 -- 172.21.15.72:0/2384986748 wait complete. 2023-12-12T09:53:21.737 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:21.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.955+0000 7fcacf7f5640 1 Processor -- start 2023-12-12T09:53:21.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.955+0000 7fcacf7f5640 1 -- start start 2023-12-12T09:53:21.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.956+0000 7fcacf7f5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcac80ae300 0x7fcac806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.956+0000 7fcacf7f5640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac80cdaa0 con 0x7fcac806ab70 2023-12-12T09:53:21.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.956+0000 7fcacf7f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac80cdd80 con 0x7fcac815a1e0 2023-12-12T09:53:21.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.956+0000 7fcacf7f5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac80ce060 con 0x7fcac80ae300 2023-12-12T09:53:21.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcaccd90640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fcac806ab70 0x7fcac815dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42154/0 (socket says 172.21.15.72:42154) 2023-12-12T09:53:21.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcaccd90640 1 -- 172.21.15.72:0/3558304722 learned_addr learned my addr 172.21.15.72:0/3558304722 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcacdd92640 1 --2- 172.21.15.72:0/3558304722 >> v2:172.21.15.113:3300/0 conn(0x7fcac80ae300 0x7fcac806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087618868 0 0) 0x7fcac80cdaa0 con 0x7fcac806ab70 2023-12-12T09:53:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab4005730 con 0x7fcac806ab70 2023-12-12T09:53:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1985806125 0 0) 0x7fcab4005730 con 0x7fcac806ab70 2023-12-12T09:53:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 >> v2:172.21.15.113:3300/0 conn(0x7fcac80ae300 msgr2=0x7fcac806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 --2- 172.21.15.72:0/3558304722 >> v2:172.21.15.113:3300/0 conn(0x7fcac80ae300 0x7fcac806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 >> v1:172.21.15.113:6789/0 conn(0x7fcac815a1e0 legacy=0x7fcac806a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80ce340 con 0x7fcac806ab70 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fcab8001670 con 0x7fcac806ab70 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 >> v1:172.21.15.72:6789/0 conn(0x7fcac806ab70 legacy=0x7fcac815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.957+0000 7fcabffff640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcab4005bd0 0x7fcab4005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcabffff640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcab40069d0 0x7fcab4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcabffff640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcab40074b0 0x7fcab400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80ce060 con 0x7fcab40069d0 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab400e3b0 con 0x7fcab4005bd0 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab400e570 con 0x7fcab40074b0 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcaccd90640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcab40069d0 0x7fcab4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcacd591640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcab4005bd0 0x7fcab4005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcaccd90640 1 -- 172.21.15.72:0/3558304722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcab40074b0 msgr2=0x7fcab400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcaccd90640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcab40074b0 0x7fcab400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcaccd90640 1 -- 172.21.15.72:0/3558304722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcab4005bd0 msgr2=0x7fcab4005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcaccd90640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcab4005bd0 0x7fcab4005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcaccd90640 1 -- 172.21.15.72:0/3558304722 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4005730 con 0x7fcab40069d0 2023-12-12T09:53:21.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.958+0000 7fcacdd92640 1 -- 172.21.15.72:0/3558304722 reap_dead start 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.959+0000 7fcaccd90640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcab40069d0 0x7fcab4006e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c0e0 tx=0x7fcab4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7456fea04bb70 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.959+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8005ad0 con 0x7fcab40069d0 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8005c50 con 0x7fcab40069d0 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcabffff640 1 -- 172.21.15.72:0/3558304722 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8007070 con 0x7fcab40069d0 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3558304722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcab40069d0 msgr2=0x7fcab4006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcacf7f5640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcab40069d0 0x7fcab4006e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c0e0 tx=0x7fcab4007a00 comp rx=0 tx=0).stop 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3558304722 shutdown_connections 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcacf7f5640 1 --2- 172.21.15.72:0/3558304722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcab40069d0 0x7fcab4006e50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3558304722 >> 172.21.15.72:0/3558304722 conn(0x7fcac80cee70 msgr2=0x7fcac80cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3558304722 shutdown_connections 2023-12-12T09:53:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.960+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3558304722 wait complete. 2023-12-12T09:53:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.962+0000 7fcacf7f5640 1 Processor -- start 2023-12-12T09:53:21.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.962+0000 7fcacf7f5640 1 -- start start 2023-12-12T09:53:21.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacf7f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcab40069d0 0x7fcac80b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacf7f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcac81561b0 0x7fcac80b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacf7f5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac8150010 0x7fcac80c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacf7f5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80b36c0 con 0x7fcac8150010 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacf7f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80b2a40 con 0x7fcac81561b0 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacf7f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80b2d20 con 0x7fcab40069d0 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacdd92640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac8150010 0x7fcac80c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacdd92640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac8150010 0x7fcac80c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41960/0 (socket says 172.21.15.72:41960) 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.963+0000 7fcacdd92640 1 -- 172.21.15.72:0/3485923442 learned_addr learned my addr 172.21.15.72:0/3485923442 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.964+0000 7fcacd591640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcab40069d0 0x7fcac80b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.964+0000 7fcaccd90640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcac81561b0 0x7fcac80b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.964+0000 7fcacdd92640 1 -- 172.21.15.72:0/3485923442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcab40069d0 msgr2=0x7fcac80b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.964+0000 7fcacdd92640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcab40069d0 0x7fcac80b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.964+0000 7fcacdd92640 1 -- 172.21.15.72:0/3485923442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcac81561b0 msgr2=0x7fcac80b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.964+0000 7fcacdd92640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcac81561b0 0x7fcac80b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.964+0000 7fcacdd92640 1 -- 172.21.15.72:0/3485923442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80b3000 con 0x7fcac8150010 2023-12-12T09:53:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.964+0000 7fcacdd92640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac8150010 0x7fcac80c70f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcac80c2760 tx=0x7fcac000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.965+0000 7fcabdffb640 1 -- 172.21.15.72:0/3485923442 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0010040 con 0x7fcac8150010 2023-12-12T09:53:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.965+0000 7fcabdffb640 1 -- 172.21.15.72:0/3485923442 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac000bce0 con 0x7fcac8150010 2023-12-12T09:53:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.965+0000 7fcabdffb640 1 -- 172.21.15.72:0/3485923442 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0014640 con 0x7fcac8150010 2023-12-12T09:53:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.965+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac80b32e0 con 0x7fcac8150010 2023-12-12T09:53:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.965+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac81504c0 con 0x7fcac8150010 2023-12-12T09:53:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.966+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac80b3800 con 0x7fcac8150010 2023-12-12T09:53:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.967+0000 7fcabdffb640 1 -- 172.21.15.72:0/3485923442 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcac00147a0 con 0x7fcac8150010 2023-12-12T09:53:21.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.968+0000 7fcabdffb640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcaac06ae70 0x7fcaac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:21.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.968+0000 7fcabdffb640 1 -- 172.21.15.72:0/3485923442 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcac0089df0 con 0x7fcac8150010 2023-12-12T09:53:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.968+0000 7fcacd591640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcaac06ae70 0x7fcaac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:21.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.969+0000 7fcacd591640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcaac06ae70 0x7fcaac06d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcac4001710 tx=0x7fcac4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:21.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:21.972+0000 7fcabdffb640 1 -- 172.21.15.72:0/3485923442 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcac80b3800 con 0x7fcac8150010 2023-12-12T09:53:22.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.164+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcac80b3800 con 0x7fcac8150010 2023-12-12T09:53:22.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.165+0000 7fcabdffb640 1 -- 172.21.15.72:0/3485923442 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcac80b3800 con 0x7fcac8150010 2023-12-12T09:53:22.167 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:22.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcaac06ae70 msgr2=0x7fcaac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcaac06ae70 0x7fcaac06d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcac4001710 tx=0x7fcac4009040 comp rx=0 tx=0).stop 2023-12-12T09:53:22.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac8150010 msgr2=0x7fcac80c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac8150010 0x7fcac80c70f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcac80c2760 tx=0x7fcac000b480 comp rx=0 tx=0).stop 2023-12-12T09:53:22.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 shutdown_connections 2023-12-12T09:53:22.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcaac06ae70 0x7fcaac06d330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac8150010 0x7fcac80c70f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcac81561b0 0x7fcac80b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 --2- 172.21.15.72:0/3485923442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcab40069d0 0x7fcac80b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 >> 172.21.15.72:0/3485923442 conn(0x7fcac80cee70 msgr2=0x7fcac80b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:22.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 shutdown_connections 2023-12-12T09:53:22.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.168+0000 7fcacf7f5640 1 -- 172.21.15.72:0/3485923442 wait complete. 2023-12-12T09:53:22.189 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:22.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.403+0000 7f53d31f1640 1 Processor -- start 2023-12-12T09:53:22.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.403+0000 7f53d31f1640 1 -- start start 2023-12-12T09:53:22.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.405+0000 7f53d31f1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.405+0000 7f53d31f1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f53cc141790 0x7f53cc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.405+0000 7f53d31f1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc15b780 con 0x7f53cc06d770 2023-12-12T09:53:22.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.405+0000 7f53d31f1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53cc141c20 con 0x7f53cc141790 2023-12-12T09:53:22.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.405+0000 7f53d31f1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0b2510 con 0x7f53cc06db50 2023-12-12T09:53:22.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.406+0000 7f53cbfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.406+0000 7f53cbfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:39332/0 (socket says 172.21.15.72:39332) 2023-12-12T09:53:22.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.406+0000 7f53cbfff640 1 -- 172.21.15.72:0/2901294209 learned_addr learned my addr 172.21.15.72:0/2901294209 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:22.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.406+0000 7f53d178e640 1 --2- 172.21.15.72:0/2901294209 >> v2:172.21.15.113:3300/0 conn(0x7f53cc141790 0x7f53cc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.406+0000 7f53cbfff640 1 -- 172.21.15.72:0/2901294209 >> v2:172.21.15.113:3300/0 conn(0x7f53cc141790 msgr2=0x7f53cc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:22.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.406+0000 7f53cbfff640 1 --2- 172.21.15.72:0/2901294209 >> v2:172.21.15.113:3300/0 conn(0x7f53cc141790 0x7f53cc0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.406+0000 7f53cbfff640 1 -- 172.21.15.72:0/2901294209 >> v1:172.21.15.113:6789/0 conn(0x7f53cc06d770 legacy=0x7f53cc0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.406+0000 7f53cbfff640 1 -- 172.21.15.72:0/2901294209 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc15b9d0 con 0x7f53cc06db50 2023-12-12T09:53:22.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.408+0000 7f53cbfff640 1 --2- 172.21.15.72:0/2901294209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc141250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009870 tx=0x7f53b802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59786617f90d2d4f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:22.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.408+0000 7f53cb7fe640 1 -- 172.21.15.72:0/2901294209 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8030840 con 0x7f53cc06db50 2023-12-12T09:53:22.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53cb7fe640 1 -- 172.21.15.72:0/2901294209 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b80309a0 con 0x7f53cc06db50 2023-12-12T09:53:22.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53cb7fe640 1 -- 172.21.15.72:0/2901294209 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8030d30 con 0x7f53cc06db50 2023-12-12T09:53:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53d31f1640 1 -- 172.21.15.72:0/2901294209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 msgr2=0x7f53cc141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53d31f1640 1 --2- 172.21.15.72:0/2901294209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc141250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009870 tx=0x7f53b802b980 comp rx=0 tx=0).stop 2023-12-12T09:53:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53d31f1640 1 -- 172.21.15.72:0/2901294209 shutdown_connections 2023-12-12T09:53:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53d31f1640 1 --2- 172.21.15.72:0/2901294209 >> v2:172.21.15.113:3300/0 conn(0x7f53cc141790 0x7f53cc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53d31f1640 1 --2- 172.21.15.72:0/2901294209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc141250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53d31f1640 1 -- 172.21.15.72:0/2901294209 >> 172.21.15.72:0/2901294209 conn(0x7f53cc0cb850 msgr2=0x7f53cc0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53d31f1640 1 -- 172.21.15.72:0/2901294209 shutdown_connections 2023-12-12T09:53:22.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.409+0000 7f53d31f1640 1 -- 172.21.15.72:0/2901294209 wait complete. 2023-12-12T09:53:22.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.411+0000 7f53d31f1640 1 Processor -- start 2023-12-12T09:53:22.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.411+0000 7f53d31f1640 1 -- start start 2023-12-12T09:53:22.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.412+0000 7f53d31f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53cc06d770 0x7f53cc13ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.412+0000 7f53d31f1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc1e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.412+0000 7f53d31f1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53cc141790 0x7f53cc15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.412+0000 7f53d31f1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc1f3290 con 0x7f53cc141790 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.412+0000 7f53d31f1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc1f40f0 con 0x7f53cc06d770 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.412+0000 7f53d31f1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc1f34e0 con 0x7f53cc06db50 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.412+0000 7f53d178e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53cc141790 0x7f53cc15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d178e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53cc141790 0x7f53cc15ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41972/0 (socket says 172.21.15.72:41972) 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d178e640 1 -- 172.21.15.72:0/134901556 learned_addr learned my addr 172.21.15.72:0/134901556 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53cbfff640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc1e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d0f8d640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53cc06d770 0x7f53cc13ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d178e640 1 -- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 msgr2=0x7f53cc1e55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d178e640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc1e55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d178e640 1 -- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53cc06d770 msgr2=0x7f53cc13ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d178e640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53cc06d770 0x7f53cc13ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d178e640 1 -- 172.21.15.72:0/134901556 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b80094d0 con 0x7f53cc141790 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53cbfff640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc1e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:53:22.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.413+0000 7f53d178e640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53cc141790 0x7f53cc15ec60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f53c400efc0 tx=0x7f53c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.414+0000 7f53c97fa640 1 -- 172.21.15.72:0/134901556 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4009ba0 con 0x7f53cc141790 2023-12-12T09:53:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.414+0000 7f53c97fa640 1 -- 172.21.15.72:0/134901556 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400f040 con 0x7f53cc141790 2023-12-12T09:53:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.414+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc1f37c0 con 0x7f53cc141790 2023-12-12T09:53:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.414+0000 7f53c97fa640 1 -- 172.21.15.72:0/134901556 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c40146c0 con 0x7f53cc141790 2023-12-12T09:53:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.414+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc1f3d80 con 0x7f53cc141790 2023-12-12T09:53:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.415+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc0b4090 con 0x7f53cc141790 2023-12-12T09:53:22.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.416+0000 7f53c97fa640 1 -- 172.21.15.72:0/134901556 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53c4014820 con 0x7f53cc141790 2023-12-12T09:53:22.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.417+0000 7f53c97fa640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f53b006ae70 0x7f53b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.417+0000 7f53c97fa640 1 -- 172.21.15.72:0/134901556 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f53c408a530 con 0x7f53cc141790 2023-12-12T09:53:22.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.417+0000 7f53d0f8d640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f53b006ae70 0x7f53b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.418+0000 7f53d0f8d640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f53b006ae70 0x7f53b006d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f53c0007fd0 tx=0x7f53c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:22.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.422+0000 7f53c97fa640 1 -- 172.21.15.72:0/134901556 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53c4055180 con 0x7f53cc141790 2023-12-12T09:53:22.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.613+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f53cc1f4260 con 0x7f53cc141790 2023-12-12T09:53:22.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.614+0000 7f53c97fa640 1 -- 172.21.15.72:0/134901556 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f53c40587a0 con 0x7f53cc141790 2023-12-12T09:53:22.616 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f53b006ae70 msgr2=0x7f53b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f53b006ae70 0x7f53b006d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f53c0007fd0 tx=0x7f53c0009040 comp rx=0 tx=0).stop 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53cc141790 msgr2=0x7f53cc15ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53cc141790 0x7f53cc15ec60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f53c400efc0 tx=0x7f53c400c490 comp rx=0 tx=0).stop 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 shutdown_connections 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f53b006ae70 0x7f53b006d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53cc141790 0x7f53cc15ec60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53cc06db50 0x7f53cc1e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 --2- 172.21.15.72:0/134901556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53cc06d770 0x7f53cc13ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 >> 172.21.15.72:0/134901556 conn(0x7f53cc0cb850 msgr2=0x7f53cc071360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 shutdown_connections 2023-12-12T09:53:22.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.617+0000 7f53d31f1640 1 -- 172.21.15.72:0/134901556 wait complete. 2023-12-12T09:53:22.638 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:53:22.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.850+0000 7f2e2c168640 1 Processor -- start 2023-12-12T09:53:22.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.850+0000 7f2e2c168640 1 -- start start 2023-12-12T09:53:22.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.852+0000 7f2e2c168640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e24143e00 0x7f2e240b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.852+0000 7f2e2c168640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2e2406c410 0x7f2e2415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.852+0000 7f2e2c168640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e240b3bb0 con 0x7f2e24072180 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.852+0000 7f2e2c168640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e241441e0 con 0x7f2e2406c410 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.852+0000 7f2e2c168640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24072680 con 0x7f2e24143e00 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.852+0000 7f2e2a705640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2e2406c410 0x7f2e2415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.852+0000 7f2e2a705640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2e2406c410 0x7f2e2415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41980/0 (socket says 172.21.15.72:41980) 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.852+0000 7f2e2a705640 1 -- 172.21.15.72:0/323628587 learned_addr learned my addr 172.21.15.72:0/323628587 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.853+0000 7f2e2a705640 1 -- 172.21.15.72:0/323628587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e24143e00 msgr2=0x7f2e240b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.853+0000 7f2e29f04640 1 --2- 172.21.15.72:0/323628587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e24143e00 0x7f2e240b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.853+0000 7f2e2a705640 1 --2- 172.21.15.72:0/323628587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e24143e00 0x7f2e240b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.853+0000 7f2e2a705640 1 -- 172.21.15.72:0/323628587 >> v1:172.21.15.113:6789/0 conn(0x7f2e24072180 legacy=0x7f2e240b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.853+0000 7f2e2a705640 1 -- 172.21.15.72:0/323628587 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240b1e30 con 0x7f2e2406c410 2023-12-12T09:53:22.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.853+0000 7f2e29f04640 1 --2- 172.21.15.72:0/323628587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e24143e00 0x7f2e240b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.854+0000 7f2e2a705640 1 --2- 172.21.15.72:0/323628587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e2406c410 0x7f2e2415ecb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009e50 tx=0x7f2e1c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3e69a0d01b9ea server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.854+0000 7f2e28f02640 1 -- 172.21.15.72:0/323628587 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c02ca80 con 0x7f2e2406c410 2023-12-12T09:53:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.854+0000 7f2e28f02640 1 -- 172.21.15.72:0/323628587 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c02cbe0 con 0x7f2e2406c410 2023-12-12T09:53:22.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.854+0000 7f2e28f02640 1 -- 172.21.15.72:0/323628587 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0343f0 con 0x7f2e2406c410 2023-12-12T09:53:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.855+0000 7f2e2c168640 1 -- 172.21.15.72:0/323628587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e2406c410 msgr2=0x7f2e2415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.855+0000 7f2e2c168640 1 --2- 172.21.15.72:0/323628587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e2406c410 0x7f2e2415ecb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009e50 tx=0x7f2e1c02bee0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.855+0000 7f2e2c168640 1 -- 172.21.15.72:0/323628587 shutdown_connections 2023-12-12T09:53:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.855+0000 7f2e2c168640 1 --2- 172.21.15.72:0/323628587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e2406c410 0x7f2e2415ecb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.855+0000 7f2e2c168640 1 --2- 172.21.15.72:0/323628587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e24143e00 0x7f2e240b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.855+0000 7f2e2c168640 1 -- 172.21.15.72:0/323628587 >> 172.21.15.72:0/323628587 conn(0x7f2e240b0a90 msgr2=0x7f2e240b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.855+0000 7f2e2c168640 1 -- 172.21.15.72:0/323628587 shutdown_connections 2023-12-12T09:53:22.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.855+0000 7f2e2c168640 1 -- 172.21.15.72:0/323628587 wait complete. 2023-12-12T09:53:22.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.857+0000 7f2e2c168640 1 Processor -- start 2023-12-12T09:53:22.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.857+0000 7f2e2c168640 1 -- start start 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2c168640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e2406c410 0x7f2e24135d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2c168640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e24072180 0x7f2e241f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2c168640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e24143e00 0x7f2e2413be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2c168640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2413ce80 con 0x7f2e24143e00 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2c168640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2413c8c0 con 0x7f2e24072180 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2c168640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2413cba0 con 0x7f2e2406c410 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2a705640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e24143e00 0x7f2e2413be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2a705640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e24143e00 0x7f2e2413be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41994/0 (socket says 172.21.15.72:41994) 2023-12-12T09:53:22.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.858+0000 7f2e2a705640 1 -- 172.21.15.72:0/1685457993 learned_addr learned my addr 172.21.15.72:0/1685457993 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:22.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e29f04640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e2406c410 0x7f2e24135d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e29703640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e24072180 0x7f2e241f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e2a705640 1 -- 172.21.15.72:0/1685457993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e2406c410 msgr2=0x7f2e24135d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e2a705640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e2406c410 0x7f2e24135d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e2a705640 1 -- 172.21.15.72:0/1685457993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e24072180 msgr2=0x7f2e241f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e2a705640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e24072180 0x7f2e241f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e2a705640 1 -- 172.21.15.72:0/1685457993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c009b30 con 0x7f2e24143e00 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e2a705640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e24143e00 0x7f2e2413be50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009e50 tx=0x7f2e1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e1a7fc640 1 -- 172.21.15.72:0/1685457993 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c009510 con 0x7f2e24143e00 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e1a7fc640 1 -- 172.21.15.72:0/1685457993 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c03e450 con 0x7f2e24143e00 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.859+0000 7f2e1a7fc640 1 -- 172.21.15.72:0/1685457993 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c03e800 con 0x7f2e24143e00 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.860+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e240cca00 con 0x7f2e24143e00 2023-12-12T09:53:22.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.860+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e241362d0 con 0x7f2e24143e00 2023-12-12T09:53:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.861+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e240ccec0 con 0x7f2e24143e00 2023-12-12T09:53:22.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.862+0000 7f2e1a7fc640 1 -- 172.21.15.72:0/1685457993 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e1c02cd50 con 0x7f2e24143e00 2023-12-12T09:53:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.862+0000 7f2e1a7fc640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2e0806ae40 0x7f2e0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.863+0000 7f2e1a7fc640 1 -- 172.21.15.72:0/1685457993 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2e1c0ac5e0 con 0x7f2e24143e00 2023-12-12T09:53:22.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.863+0000 7f2e29f04640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2e0806ae40 0x7f2e0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:22.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.863+0000 7f2e29f04640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2e0806ae40 0x7f2e0806d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000a950 tx=0x7f2e20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:22.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:22.867+0000 7f2e1a7fc640 1 -- 172.21.15.72:0/1685457993 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e1c077370 con 0x7f2e24143e00 2023-12-12T09:53:23.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.070+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e240745d0 con 0x7f2e24143e00 2023-12-12T09:53:23.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.070+0000 7f2e1a7fc640 1 -- 172.21.15.72:0/1685457993 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2e1c07a990 con 0x7f2e24143e00 2023-12-12T09:53:23.072 INFO:teuthology.orchestra.run.smithi072.stderr:listed 0 entries 2023-12-12T09:53:23.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2e0806ae40 msgr2=0x7f2e0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:23.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2e0806ae40 0x7f2e0806d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000a950 tx=0x7f2e20008040 comp rx=0 tx=0).stop 2023-12-12T09:53:23.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e24143e00 msgr2=0x7f2e2413be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:23.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e24143e00 0x7f2e2413be50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009e50 tx=0x7f2e1c004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:23.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 shutdown_connections 2023-12-12T09:53:23.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2e0806ae40 0x7f2e0806d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e24143e00 0x7f2e2413be50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e24072180 0x7f2e241f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 --2- 172.21.15.72:0/1685457993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e2406c410 0x7f2e24135d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 >> 172.21.15.72:0/1685457993 conn(0x7f2e240b0a90 msgr2=0x7f2e240cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:23.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 shutdown_connections 2023-12-12T09:53:23.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.073+0000 7f2e2c168640 1 -- 172.21.15.72:0/1685457993 wait complete. 2023-12-12T09:53:23.094 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-12-12T09:53:23.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:23.095 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-12-12T09:53:23.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:53:23.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:23.206 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-12T09:53:23.268 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:53:23.268 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:23.326 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:53:23.327 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:23.379 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:53:23.379 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:23.429 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:23.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.680+0000 7f773be56640 1 Processor -- start 2023-12-12T09:53:23.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.680+0000 7f773be56640 1 -- start start 2023-12-12T09:53:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.681+0000 7f773be56640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7734143e00 0x7f77340b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.681+0000 7f773be56640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7734072180 0x7f77340b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.681+0000 7f773be56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406c410 0x7f77340b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.681+0000 7f773be56640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77340b3bb0 con 0x7f7734072180 2023-12-12T09:53:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.681+0000 7f773be56640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77341441e0 con 0x7f7734143e00 2023-12-12T09:53:23.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.681+0000 7f773be56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7734072680 con 0x7f773406c410 2023-12-12T09:53:23.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f77393f1640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7734072180 0x7f77340b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:23.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f77393f1640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7734072180 0x7f77340b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42010/0 (socket says 172.21.15.72:42010) 2023-12-12T09:53:23.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f77393f1640 1 -- 172.21.15.72:0/4183390779 learned_addr learned my addr 172.21.15.72:0/4183390779 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f773a3f3640 1 --2- 172.21.15.72:0/4183390779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406c410 0x7f77340b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f7739bf2640 1 --2- 172.21.15.72:0/4183390779 >> v2:172.21.15.113:3300/0 conn(0x7f7734143e00 0x7f77340b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f77393f1640 1 -- 172.21.15.72:0/4183390779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406c410 msgr2=0x7f77340b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f77393f1640 1 --2- 172.21.15.72:0/4183390779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406c410 0x7f77340b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f77393f1640 1 -- 172.21.15.72:0/4183390779 >> v2:172.21.15.113:3300/0 conn(0x7f7734143e00 msgr2=0x7f77340b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f77393f1640 1 --2- 172.21.15.72:0/4183390779 >> v2:172.21.15.113:3300/0 conn(0x7f7734143e00 0x7f77340b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f77393f1640 1 -- 172.21.15.72:0/4183390779 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77340b1e30 con 0x7f7734072180 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f7739bf2640 1 --2- 172.21.15.72:0/4183390779 >> v2:172.21.15.113:3300/0 conn(0x7f7734143e00 0x7f77340b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.682+0000 7f773a3f3640 1 --2- 172.21.15.72:0/4183390779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406c410 0x7f77340b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.683+0000 7f77393f1640 1 --2- 172.21.15.72:0/4183390779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f77340b4580 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f773406b760 tx=0x7f772402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a43c59e25b9530a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:23.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f7738bf0640 1 -- 172.21.15.72:0/4183390779 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724031870 con 0x7f7734072180 2023-12-12T09:53:23.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f7738bf0640 1 -- 172.21.15.72:0/4183390779 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77240319d0 con 0x7f7734072180 2023-12-12T09:53:23.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f7738bf0640 1 -- 172.21.15.72:0/4183390779 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724031d60 con 0x7f7734072180 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 -- 172.21.15.72:0/4183390779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 msgr2=0x7f77340b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 --2- 172.21.15.72:0/4183390779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f77340b4580 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f773406b760 tx=0x7f772402b910 comp rx=0 tx=0).stop 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 -- 172.21.15.72:0/4183390779 shutdown_connections 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 --2- 172.21.15.72:0/4183390779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406c410 0x7f77340b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 --2- 172.21.15.72:0/4183390779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f77340b4580 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 --2- 172.21.15.72:0/4183390779 >> v2:172.21.15.113:3300/0 conn(0x7f7734143e00 0x7f77340b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 -- 172.21.15.72:0/4183390779 >> 172.21.15.72:0/4183390779 conn(0x7f77340b0a90 msgr2=0x7f77340b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 -- 172.21.15.72:0/4183390779 shutdown_connections 2023-12-12T09:53:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.684+0000 7f773be56640 1 -- 172.21.15.72:0/4183390779 wait complete. 2023-12-12T09:53:23.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.686+0000 7f773be56640 1 Processor -- start 2023-12-12T09:53:23.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.686+0000 7f773be56640 1 -- start start 2023-12-12T09:53:23.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f773be56640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f773406c410 0x7f773415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:23.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f773be56640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f773413d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:23.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f773be56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7734143e00 0x7f77340cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f773be56640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77341f5f40 con 0x7f7734072180 2023-12-12T09:53:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f773be56640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77341f6770 con 0x7f773406c410 2023-12-12T09:53:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f773be56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773413df70 con 0x7f7734143e00 2023-12-12T09:53:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f77393f1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f773413d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f77393f1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f773413d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42026/0 (socket says 172.21.15.72:42026) 2023-12-12T09:53:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.687+0000 7f77393f1640 1 -- 172.21.15.72:0/2886950085 learned_addr learned my addr 172.21.15.72:0/2886950085 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f7739bf2640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f773406c410 0x7f773415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f773a3f3640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7734143e00 0x7f77340cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f77393f1640 1 -- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7734143e00 msgr2=0x7f77340cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f77393f1640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7734143e00 0x7f77340cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f77393f1640 1 -- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f773406c410 msgr2=0x7f773415aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f77393f1640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f773406c410 0x7f773415aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f77393f1640 1 -- 172.21.15.72:0/2886950085 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77240094d0 con 0x7f7734072180 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f773a3f3640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7734143e00 0x7f77340cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f7739bf2640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f773406c410 0x7f773415aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.688+0000 7f77393f1640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f773413d470 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7724000c00 tx=0x7f77240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.689+0000 7f772a7fc640 1 -- 172.21.15.72:0/2886950085 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772403adf0 con 0x7f7734072180 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.689+0000 7f772a7fc640 1 -- 172.21.15.72:0/2886950085 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77240443f0 con 0x7f7734072180 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.689+0000 7f772a7fc640 1 -- 172.21.15.72:0/2886950085 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77240447a0 con 0x7f7734072180 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.689+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77341f6120 con 0x7f7734072180 2023-12-12T09:53:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.689+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77341f37f0 con 0x7f7734072180 2023-12-12T09:53:23.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.690+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77341f68e0 con 0x7f7734072180 2023-12-12T09:53:23.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.691+0000 7f772a7fc640 1 -- 172.21.15.72:0/2886950085 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f772404d450 con 0x7f7734072180 2023-12-12T09:53:23.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.692+0000 7f772a7fc640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f771806ae40 0x7f771806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:23.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.692+0000 7f772a7fc640 1 -- 172.21.15.72:0/2886950085 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f772402d040 con 0x7f7734072180 2023-12-12T09:53:23.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.692+0000 7f7739bf2640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f771806ae40 0x7f771806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:23.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.693+0000 7f7739bf2640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f771806ae40 0x7f771806d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7730006fd0 tx=0x7f7730008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:23.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.697+0000 7f772a7fc640 1 -- 172.21.15.72:0/2886950085 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7724076d30 con 0x7f7734072180 2023-12-12T09:53:23.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.888+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77340745d0 con 0x7f7734072180 2023-12-12T09:53:23.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.891+0000 7f772a7fc640 1 -- 172.21.15.72:0/2886950085 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7f7724035090 con 0x7f7734072180 2023-12-12T09:53:23.893 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:23.893 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:23.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.893+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f771806ae40 msgr2=0x7f771806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.893+0000 7f773be56640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f771806ae40 0x7f771806d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7730006fd0 tx=0x7f7730008040 comp rx=0 tx=0).stop 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.893+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 msgr2=0x7f773413d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.893+0000 7f773be56640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f773413d470 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7724000c00 tx=0x7f77240046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.894+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 shutdown_connections 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.894+0000 7f773be56640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f771806ae40 0x7f771806d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.894+0000 7f773be56640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7734143e00 0x7f77340cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.894+0000 7f773be56640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7734072180 0x7f773413d470 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.894+0000 7f773be56640 1 --2- 172.21.15.72:0/2886950085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f773406c410 0x7f773415aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.894+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 >> 172.21.15.72:0/2886950085 conn(0x7f77340b0a90 msgr2=0x7f77340ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.894+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 shutdown_connections 2023-12-12T09:53:23.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:23.894+0000 7f773be56640 1 -- 172.21.15.72:0/2886950085 wait complete. 2023-12-12T09:53:23.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:23.914 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:24.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.126+0000 7fd577017640 1 Processor -- start 2023-12-12T09:53:24.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.127+0000 7fd577017640 1 -- start start 2023-12-12T09:53:24.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd577017640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd5700b4550 0x7fd5700cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd577017640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5700d0140 0x7fd57015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd577017640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5700ce6e0 con 0x7fd57015e720 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd577017640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5700d21f0 con 0x7fd5700b4550 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd577017640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5700ce9c0 con 0x7fd5700d0140 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd574db3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd5700b4550 0x7fd5700cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd574db3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd5700b4550 0x7fd5700cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42028/0 (socket says 172.21.15.72:42028) 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd56ffff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd57015e720 0x7fd5700cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42170/0 (socket says 172.21.15.72:42170) 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.128+0000 7fd574db3640 1 -- 172.21.15.72:0/2079892044 learned_addr learned my addr 172.21.15.72:0/2079892044 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.129+0000 7fd5755b4640 1 --2- 172.21.15.72:0/2079892044 >> v2:172.21.15.113:3300/0 conn(0x7fd5700d0140 0x7fd57015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.129+0000 7fd574db3640 1 -- 172.21.15.72:0/2079892044 >> v2:172.21.15.113:3300/0 conn(0x7fd5700d0140 msgr2=0x7fd57015cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.129+0000 7fd574db3640 1 --2- 172.21.15.72:0/2079892044 >> v2:172.21.15.113:3300/0 conn(0x7fd5700d0140 0x7fd57015cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.129+0000 7fd574db3640 1 -- 172.21.15.72:0/2079892044 >> v1:172.21.15.72:6789/0 conn(0x7fd57015e720 legacy=0x7fd5700cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.129+0000 7fd574db3640 1 -- 172.21.15.72:0/2079892044 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700ceca0 con 0x7fd5700b4550 2023-12-12T09:53:24.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.130+0000 7fd574db3640 1 --2- 172.21.15.72:0/2079892044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd5700cdc70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd57008d7c0 tx=0x7fd56402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05451069dd88170 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:24.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.130+0000 7fd56f7fe640 1 -- 172.21.15.72:0/2079892044 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564030860 con 0x7fd5700b4550 2023-12-12T09:53:24.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.130+0000 7fd56f7fe640 1 -- 172.21.15.72:0/2079892044 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5640309c0 con 0x7fd5700b4550 2023-12-12T09:53:24.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.130+0000 7fd56f7fe640 1 -- 172.21.15.72:0/2079892044 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564030d50 con 0x7fd5700b4550 2023-12-12T09:53:24.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.131+0000 7fd577017640 1 -- 172.21.15.72:0/2079892044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 msgr2=0x7fd5700cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.131+0000 7fd577017640 1 --2- 172.21.15.72:0/2079892044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd5700cdc70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd57008d7c0 tx=0x7fd56402b9a0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.131+0000 7fd577017640 1 -- 172.21.15.72:0/2079892044 shutdown_connections 2023-12-12T09:53:24.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.131+0000 7fd577017640 1 --2- 172.21.15.72:0/2079892044 >> v2:172.21.15.113:3300/0 conn(0x7fd5700d0140 0x7fd57015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.131+0000 7fd577017640 1 --2- 172.21.15.72:0/2079892044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd5700cdc70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.131+0000 7fd577017640 1 -- 172.21.15.72:0/2079892044 >> 172.21.15.72:0/2079892044 conn(0x7fd57009c430 msgr2=0x7fd5700d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:24.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.131+0000 7fd577017640 1 -- 172.21.15.72:0/2079892044 shutdown_connections 2023-12-12T09:53:24.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.131+0000 7fd577017640 1 -- 172.21.15.72:0/2079892044 wait complete. 2023-12-12T09:53:24.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.133+0000 7fd577017640 1 Processor -- start 2023-12-12T09:53:24.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.133+0000 7fd577017640 1 -- start start 2023-12-12T09:53:24.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd577017640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd570154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd577017640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5700d0140 0x7fd5700ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd577017640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd57015e720 0x7fd57015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd577017640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701553d0 con 0x7fd5700b4550 2023-12-12T09:53:24.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd577017640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700cb620 con 0x7fd57015e720 2023-12-12T09:53:24.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd577017640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700caa10 con 0x7fd5700d0140 2023-12-12T09:53:24.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd574db3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd570154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd574db3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd570154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42044/0 (socket says 172.21.15.72:42044) 2023-12-12T09:53:24.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.134+0000 7fd574db3640 1 -- 172.21.15.72:0/149803179 learned_addr learned my addr 172.21.15.72:0/149803179 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd56ffff640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5700d0140 0x7fd5700ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd574db3640 1 -- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5700d0140 msgr2=0x7fd5700ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd5755b4640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd57015e720 0x7fd57015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd574db3640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5700d0140 0x7fd5700ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd574db3640 1 -- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd57015e720 msgr2=0x7fd57015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd574db3640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd57015e720 0x7fd57015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd574db3640 1 -- 172.21.15.72:0/149803179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640094d0 con 0x7fd5700b4550 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd56ffff640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5700d0140 0x7fd5700ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:24.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd574db3640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd570154e90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd56402b9e0 tx=0x7fd564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:24.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.135+0000 7fd56d7fa640 1 -- 172.21.15.72:0/149803179 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564039d60 con 0x7fd5700b4550 2023-12-12T09:53:24.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.136+0000 7fd56d7fa640 1 -- 172.21.15.72:0/149803179 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd564043400 con 0x7fd5700b4550 2023-12-12T09:53:24.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.136+0000 7fd56d7fa640 1 -- 172.21.15.72:0/149803179 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640437b0 con 0x7fd5700b4550 2023-12-12T09:53:24.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.136+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5700cacf0 con 0x7fd5700b4550 2023-12-12T09:53:24.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.136+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5700cb2b0 con 0x7fd5700b4550 2023-12-12T09:53:24.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.137+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57015ed10 con 0x7fd5700b4550 2023-12-12T09:53:24.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.138+0000 7fd56d7fa640 1 -- 172.21.15.72:0/149803179 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd564043910 con 0x7fd5700b4550 2023-12-12T09:53:24.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.138+0000 7fd56d7fa640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd55c06ae40 0x7fd55c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.139+0000 7fd56d7fa640 1 -- 172.21.15.72:0/149803179 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd56403e070 con 0x7fd5700b4550 2023-12-12T09:53:24.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.139+0000 7fd56ffff640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd55c06ae40 0x7fd55c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.139+0000 7fd56ffff640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd55c06ae40 0x7fd55c06d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5700b8d00 tx=0x7fd560009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:24.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.143+0000 7fd56d7fa640 1 -- 172.21.15.72:0/149803179 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd564077870 con 0x7fd5700b4550 2023-12-12T09:53:24.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.333+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd570074450 con 0x7fd5700b4550 2023-12-12T09:53:24.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.334+0000 7fd56d7fa640 1 -- 172.21.15.72:0/149803179 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5640340a0 con 0x7fd5700b4550 2023-12-12T09:53:24.336 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:24.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.336+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd55c06ae40 msgr2=0x7fd55c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.336+0000 7fd577017640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd55c06ae40 0x7fd55c06d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5700b8d00 tx=0x7fd560009040 comp rx=0 tx=0).stop 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.336+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 msgr2=0x7fd570154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.336+0000 7fd577017640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd570154e90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd56402b9e0 tx=0x7fd564004730 comp rx=0 tx=0).stop 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.336+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 shutdown_connections 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.337+0000 7fd577017640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd55c06ae40 0x7fd55c06d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.337+0000 7fd577017640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd57015e720 0x7fd57015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.337+0000 7fd577017640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5700d0140 0x7fd5700ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.337+0000 7fd577017640 1 --2- 172.21.15.72:0/149803179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5700b4550 0x7fd570154e90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.337+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 >> 172.21.15.72:0/149803179 conn(0x7fd57009c430 msgr2=0x7fd57006c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.337+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 shutdown_connections 2023-12-12T09:53:24.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.337+0000 7fd577017640 1 -- 172.21.15.72:0/149803179 wait complete. 2023-12-12T09:53:24.357 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:24.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.571+0000 7f13a7fff640 1 Processor -- start 2023-12-12T09:53:24.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.571+0000 7f13a7fff640 1 -- start start 2023-12-12T09:53:24.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.572+0000 7f13a7fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f13a8143a40 0x7f13a80cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.572+0000 7f13a7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a80ce2b0 0x7f13a80cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.572+0000 7f13a7fff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a80cc0a0 con 0x7f13a8143660 2023-12-12T09:53:24.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.572+0000 7f13a7fff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a80cc310 con 0x7f13a8143a40 2023-12-12T09:53:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.572+0000 7f13a7fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80cc5f0 con 0x7f13a80ce2b0 2023-12-12T09:53:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.573+0000 7f13a6ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f13a8143660 0x7f13a80cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42184/0 (socket says 172.21.15.72:42184) 2023-12-12T09:53:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.573+0000 7f13a6ffd640 1 -- 172.21.15.72:0/2381721877 learned_addr learned my addr 172.21.15.72:0/2381721877 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.573+0000 7f13a67fc640 1 --2- 172.21.15.72:0/2381721877 >> v2:172.21.15.72:3300/0 conn(0x7f13a8143a40 0x7f13a80cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.573+0000 7f13a67fc640 1 -- 172.21.15.72:0/2381721877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a80ce2b0 msgr2=0x7f13a80cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.573+0000 7f13a67fc640 1 --2- 172.21.15.72:0/2381721877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a80ce2b0 0x7f13a80cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.573+0000 7f13a67fc640 1 -- 172.21.15.72:0/2381721877 >> v1:172.21.15.72:6789/0 conn(0x7f13a8143660 legacy=0x7f13a80cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.573+0000 7f13a67fc640 1 -- 172.21.15.72:0/2381721877 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a8070180 con 0x7f13a8143a40 2023-12-12T09:53:24.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.574+0000 7f13a67fc640 1 --2- 172.21.15.72:0/2381721877 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143a40 0x7f13a80cb620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1390009dc0 tx=0x7f139002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc41dd7f5f78f37 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:24.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.574+0000 7f13a5ffb640 1 -- 172.21.15.72:0/2381721877 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139002cc60 con 0x7f13a8143a40 2023-12-12T09:53:24.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a5ffb640 1 -- 172.21.15.72:0/2381721877 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139002cdc0 con 0x7f13a8143a40 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a5ffb640 1 -- 172.21.15.72:0/2381721877 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390034660 con 0x7f13a8143a40 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a7fff640 1 -- 172.21.15.72:0/2381721877 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143a40 msgr2=0x7f13a80cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a7fff640 1 --2- 172.21.15.72:0/2381721877 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143a40 0x7f13a80cb620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1390009dc0 tx=0x7f139002beb0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a7fff640 1 -- 172.21.15.72:0/2381721877 shutdown_connections 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a7fff640 1 --2- 172.21.15.72:0/2381721877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a80ce2b0 0x7f13a80cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a7fff640 1 --2- 172.21.15.72:0/2381721877 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143a40 0x7f13a80cb620 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a7fff640 1 -- 172.21.15.72:0/2381721877 >> 172.21.15.72:0/2381721877 conn(0x7f13a80b0940 msgr2=0x7f13a80b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a7fff640 1 -- 172.21.15.72:0/2381721877 shutdown_connections 2023-12-12T09:53:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.575+0000 7f13a7fff640 1 -- 172.21.15.72:0/2381721877 wait complete. 2023-12-12T09:53:24.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.577+0000 7f13a7fff640 1 Processor -- start 2023-12-12T09:53:24.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.577+0000 7f13a7fff640 1 -- start start 2023-12-12T09:53:24.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a80ce2b0 0x7f13a80ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143660 0x7f13a809a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a8143a40 0x7f13a809c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a7fff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a809bde0 con 0x7f13a8143660 2023-12-12T09:53:24.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a7fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a809ace0 con 0x7f13a8143a40 2023-12-12T09:53:24.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a7fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a8164c20 con 0x7f13a80ce2b0 2023-12-12T09:53:24.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a67fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143660 0x7f13a809a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a67fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143660 0x7f13a809a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42076/0 (socket says 172.21.15.72:42076) 2023-12-12T09:53:24.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.578+0000 7f13a67fc640 1 -- 172.21.15.72:0/1441111408 learned_addr learned my addr 172.21.15.72:0/1441111408 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:24.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f13a67fc640 1 -- 172.21.15.72:0/1441111408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a80ce2b0 msgr2=0x7f13a80ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f13a67fc640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a80ce2b0 0x7f13a80ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f13a67fc640 1 -- 172.21.15.72:0/1441111408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a8143a40 msgr2=0x7f13a809c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f13a67fc640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a8143a40 0x7f13a809c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f13a67fc640 1 -- 172.21.15.72:0/1441111408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1390009a70 con 0x7f13a8143660 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f13a67fc640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143660 0x7f13a809a3d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1390037040 tx=0x7f1390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f139f7fe640 1 -- 172.21.15.72:0/1441111408 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139003e610 con 0x7f13a8143660 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f139f7fe640 1 -- 172.21.15.72:0/1441111408 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139003e770 con 0x7f13a8143660 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a8164f00 con 0x7f13a8143660 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f139f7fe640 1 -- 172.21.15.72:0/1441111408 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139003eb00 con 0x7f13a8143660 2023-12-12T09:53:24.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.579+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a81654c0 con 0x7f13a8143660 2023-12-12T09:53:24.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.580+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a8067910 con 0x7f13a8143660 2023-12-12T09:53:24.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.582+0000 7f139f7fe640 1 -- 172.21.15.72:0/1441111408 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f139003b050 con 0x7f13a8143660 2023-12-12T09:53:24.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.582+0000 7f139f7fe640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138806ae70 0x7f138806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:24.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.583+0000 7f139f7fe640 1 -- 172.21.15.72:0/1441111408 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f13900ac120 con 0x7f13a8143660 2023-12-12T09:53:24.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.583+0000 7f13a6ffd640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138806ae70 0x7f138806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:24.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.583+0000 7f13a6ffd640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138806ae70 0x7f138806d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13980015f0 tx=0x7f1398007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:24.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.587+0000 7f139f7fe640 1 -- 172.21.15.72:0/1441111408 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1390076df0 con 0x7f13a8143660 2023-12-12T09:53:24.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.775+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13a81656b0 con 0x7f13a8143660 2023-12-12T09:53:24.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.776+0000 7f139f7fe640 1 -- 172.21.15.72:0/1441111408 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f139007a410 con 0x7f13a8143660 2023-12-12T09:53:24.778 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:24.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138806ae70 msgr2=0x7f138806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138806ae70 0x7f138806d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13980015f0 tx=0x7f1398007a40 comp rx=0 tx=0).stop 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143660 msgr2=0x7f13a809a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143660 0x7f13a809a3d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1390037040 tx=0x7f1390004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 shutdown_connections 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138806ae70 0x7f138806d330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13a8143a40 0x7f13a809c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a8143660 0x7f13a809a3d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 --2- 172.21.15.72:0/1441111408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a80ce2b0 0x7f13a80ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.778+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 >> 172.21.15.72:0/1441111408 conn(0x7f13a80b0940 msgr2=0x7f13a80b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.779+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 shutdown_connections 2023-12-12T09:53:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:24.779+0000 7f13a7fff640 1 -- 172.21.15.72:0/1441111408 wait complete. 2023-12-12T09:53:24.800 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:25.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.018+0000 7f43e8954640 1 Processor -- start 2023-12-12T09:53:25.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.018+0000 7f43e8954640 1 -- start start 2023-12-12T09:53:25.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e8954640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43e40ce150 0x7f43e40ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e8954640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e40cedf0 0x7f43e40cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e8954640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e40cf790 con 0x7f43e4155400 2023-12-12T09:53:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e8954640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e406c500 con 0x7f43e40ce150 2023-12-12T09:53:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e8954640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e415b4a0 con 0x7f43e40cedf0 2023-12-12T09:53:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e37fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f43e4155400 0x7f43e406bdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42194/0 (socket says 172.21.15.72:42194) 2023-12-12T09:53:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e37fe640 1 -- 172.21.15.72:0/569662154 learned_addr learned my addr 172.21.15.72:0/569662154 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e2ffd640 1 --2- 172.21.15.72:0/569662154 >> v2:172.21.15.113:3300/0 conn(0x7f43e40ce150 0x7f43e40ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e3fff640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e40cedf0 0x7f43e40cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516563726 0 0) 0x7f43e40cf790 con 0x7f43e4155400 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.020+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43cc003520 con 0x7f43e4155400 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3055955671 0 0) 0x7f43cc003520 con 0x7f43e4155400 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e40cedf0 msgr2=0x7f43e40cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e40cedf0 0x7f43e40cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 >> v2:172.21.15.113:3300/0 conn(0x7f43e40ce150 msgr2=0x7f43e40ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 --2- 172.21.15.72:0/569662154 >> v2:172.21.15.113:3300/0 conn(0x7f43e40ce150 0x7f43e40ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e40adbb0 con 0x7f43e4155400 2023-12-12T09:53:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f43d4001a10 con 0x7f43e4155400 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e3fff640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e40cedf0 0x7f43e40cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 >> v1:172.21.15.72:6789/0 conn(0x7f43e4155400 legacy=0x7f43e406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43cc003970 0x7f43cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0047c0 0x7f43cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e40cedf0 0x7f43cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e406c500 con 0x7f43cc003970 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e415b4a0 con 0x7f43cc0047c0 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc00b980 con 0x7f43e40cedf0 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.021+0000 7f43e37fe640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43cc003970 0x7f43cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.022+0000 7f43e37fe640 1 -- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e40cedf0 msgr2=0x7f43cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.022+0000 7f43e37fe640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43e40cedf0 0x7f43cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.022+0000 7f43e37fe640 1 -- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0047c0 msgr2=0x7f43cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.022+0000 7f43e37fe640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0047c0 0x7f43cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.022+0000 7f43e37fe640 1 -- 172.21.15.72:0/569662154 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc003520 con 0x7f43cc003970 2023-12-12T09:53:25.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.023+0000 7f43e37fe640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43cc003970 0x7f43cc003d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f43d400a880 tx=0x7f43cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f09bacbf366c475 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.023+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d402cea0 con 0x7f43cc003970 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.023+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d402f070 con 0x7f43cc003970 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.023+0000 7f43e27fc640 1 -- 172.21.15.72:0/569662154 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4034870 con 0x7f43cc003970 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.023+0000 7f43e8954640 1 -- 172.21.15.72:0/569662154 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43cc003970 msgr2=0x7f43cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.023+0000 7f43e8954640 1 --2- 172.21.15.72:0/569662154 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43cc003970 0x7f43cc003d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f43d400a880 tx=0x7f43cc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.023+0000 7f43e3fff640 1 -- 172.21.15.72:0/569662154 reap_dead start 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.024+0000 7f43e8954640 1 -- 172.21.15.72:0/569662154 shutdown_connections 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.024+0000 7f43e8954640 1 -- 172.21.15.72:0/569662154 >> 172.21.15.72:0/569662154 conn(0x7f43e406a640 msgr2=0x7f43e406aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.024+0000 7f43e8954640 1 -- 172.21.15.72:0/569662154 shutdown_connections 2023-12-12T09:53:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.024+0000 7f43e8954640 1 -- 172.21.15.72:0/569662154 wait complete. 2023-12-12T09:53:25.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.025+0000 7f43e8954640 1 Processor -- start 2023-12-12T09:53:25.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.026+0000 7f43e8954640 1 -- start start 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e8954640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43cc003970 0x7f43e41e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e8954640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e41e57f0 0x7f43e415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e8954640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4148810 0x7f43e4148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e8954640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41e5c70 con 0x7f43e4148810 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e8954640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41e5390 con 0x7f43e41e57f0 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e8954640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41e4cb0 con 0x7f43cc003970 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4148810 0x7f43e4148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4148810 0x7f43e4148c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44784/0 (socket says 172.21.15.72:44784) 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e3fff640 1 -- 172.21.15.72:0/3564567982 learned_addr learned my addr 172.21.15.72:0/3564567982 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e37fe640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43cc003970 0x7f43e41e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e2ffd640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e41e57f0 0x7f43e415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e3fff640 1 -- 172.21.15.72:0/3564567982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43cc003970 msgr2=0x7f43e41e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e3fff640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43cc003970 0x7f43e41e35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e3fff640 1 -- 172.21.15.72:0/3564567982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e41e57f0 msgr2=0x7f43e415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e3fff640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e41e57f0 0x7f43e415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.027+0000 7f43e3fff640 1 -- 172.21.15.72:0/3564567982 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e41e3c10 con 0x7f43e4148810 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.028+0000 7f43e3fff640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4148810 0x7f43e4148c70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f43e406d3d0 tx=0x7f43dc00e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.028+0000 7f43cbfff640 1 -- 172.21.15.72:0/3564567982 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00c070 con 0x7f43e4148810 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.028+0000 7f43cbfff640 1 -- 172.21.15.72:0/3564567982 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc004250 con 0x7f43e4148810 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.028+0000 7f43cbfff640 1 -- 172.21.15.72:0/3564567982 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc003610 con 0x7f43e4148810 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.028+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e41e3ef0 con 0x7f43e4148810 2023-12-12T09:53:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.028+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e41e44b0 con 0x7f43e4148810 2023-12-12T09:53:25.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.029+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e40ad780 con 0x7f43e4148810 2023-12-12T09:53:25.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.030+0000 7f43cbfff640 1 -- 172.21.15.72:0/3564567982 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43dc003770 con 0x7f43e4148810 2023-12-12T09:53:25.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.031+0000 7f43cbfff640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43d006ae70 0x7f43d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.031+0000 7f43cbfff640 1 -- 172.21.15.72:0/3564567982 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f43dc088e60 con 0x7f43e4148810 2023-12-12T09:53:25.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.031+0000 7f43e37fe640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43d006ae70 0x7f43d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.032+0000 7f43e37fe640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43d006ae70 0x7f43d006d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43d4034000 tx=0x7f43d4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.035+0000 7f43cbfff640 1 -- 172.21.15.72:0/3564567982 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43dc053be0 con 0x7f43e4148810 2023-12-12T09:53:25.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.223+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43e4074450 con 0x7f43e4148810 2023-12-12T09:53:25.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.224+0000 7f43cbfff640 1 -- 172.21.15.72:0/3564567982 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43dc057200 con 0x7f43e4148810 2023-12-12T09:53:25.227 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:25.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43d006ae70 msgr2=0x7f43d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43d006ae70 0x7f43d006d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43d4034000 tx=0x7f43d4009030 comp rx=0 tx=0).stop 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4148810 msgr2=0x7f43e4148c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4148810 0x7f43e4148c70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f43e406d3d0 tx=0x7f43dc00e8b0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 shutdown_connections 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43d006ae70 0x7f43d006d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4148810 0x7f43e4148c70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43e41e57f0 0x7f43e415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 --2- 172.21.15.72:0/3564567982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43cc003970 0x7f43e41e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.226+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 >> 172.21.15.72:0/3564567982 conn(0x7f43e406a640 msgr2=0x7f43e415d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:25.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.227+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 shutdown_connections 2023-12-12T09:53:25.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.227+0000 7f43e8954640 1 -- 172.21.15.72:0/3564567982 wait complete. 2023-12-12T09:53:25.245 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:25.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.450+0000 7f22a54d7640 1 Processor -- start 2023-12-12T09:53:25.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.450+0000 7f22a54d7640 1 -- start start 2023-12-12T09:53:25.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.451+0000 7f22a54d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a0143a40 0x7f22a00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.451+0000 7f22a54d7640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f22a00ce2b0 0x7f22a00cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.451+0000 7f22a54d7640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a00cc0a0 con 0x7f22a0143660 2023-12-12T09:53:25.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.451+0000 7f22a54d7640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a00cc310 con 0x7f22a00ce2b0 2023-12-12T09:53:25.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.451+0000 7f22a54d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00cc5f0 con 0x7f22a0143a40 2023-12-12T09:53:25.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f229ffff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f22a0143660 0x7f22a00cdba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39280/0 (socket says 172.21.15.72:39280) 2023-12-12T09:53:25.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f229ffff640 1 -- 172.21.15.72:0/972660059 learned_addr learned my addr 172.21.15.72:0/972660059 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:25.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f22a4cd6640 1 --2- 172.21.15.72:0/972660059 >> v2:172.21.15.72:3300/0 conn(0x7f22a00ce2b0 0x7f22a00cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f229f7fe640 1 --2- 172.21.15.72:0/972660059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a0143a40 0x7f22a00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f229effd640 1 -- 172.21.15.72:0/972660059 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197343367 0 0) 0x7f22a00cc0a0 con 0x7f22a0143660 2023-12-12T09:53:25.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f229effd640 1 -- 172.21.15.72:0/972660059 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f228c003520 con 0x7f22a0143660 2023-12-12T09:53:25.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f22a4cd6640 1 -- 172.21.15.72:0/972660059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a0143a40 msgr2=0x7f22a00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:25.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f22a4cd6640 1 --2- 172.21.15.72:0/972660059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a0143a40 0x7f22a00cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f22a4cd6640 1 -- 172.21.15.72:0/972660059 >> v1:172.21.15.72:6789/0 conn(0x7f22a0143660 legacy=0x7f22a00cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.452+0000 7f22a4cd6640 1 -- 172.21.15.72:0/972660059 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a0070180 con 0x7f22a00ce2b0 2023-12-12T09:53:25.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.453+0000 7f22a4cd6640 1 --2- 172.21.15.72:0/972660059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a00ce2b0 0x7f22a00cbb60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f22980097a0 tx=0x7f229802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859a2d9103e4b380 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f229effd640 1 -- 172.21.15.72:0/972660059 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298031830 con 0x7f22a00ce2b0 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f229effd640 1 -- 172.21.15.72:0/972660059 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2298031990 con 0x7f22a00ce2b0 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f229effd640 1 -- 172.21.15.72:0/972660059 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298031d20 con 0x7f22a00ce2b0 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f22a54d7640 1 -- 172.21.15.72:0/972660059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a00ce2b0 msgr2=0x7f22a00cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f22a54d7640 1 --2- 172.21.15.72:0/972660059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a00ce2b0 0x7f22a00cbb60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f22980097a0 tx=0x7f229802b8d0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f22a54d7640 1 -- 172.21.15.72:0/972660059 shutdown_connections 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f22a54d7640 1 --2- 172.21.15.72:0/972660059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a00ce2b0 0x7f22a00cbb60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f22a54d7640 1 --2- 172.21.15.72:0/972660059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a0143a40 0x7f22a00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f22a54d7640 1 -- 172.21.15.72:0/972660059 >> 172.21.15.72:0/972660059 conn(0x7f22a00b0940 msgr2=0x7f22a00b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f22a54d7640 1 -- 172.21.15.72:0/972660059 shutdown_connections 2023-12-12T09:53:25.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.454+0000 7f22a54d7640 1 -- 172.21.15.72:0/972660059 wait complete. 2023-12-12T09:53:25.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.456+0000 7f22a54d7640 1 Processor -- start 2023-12-12T09:53:25.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.456+0000 7f22a54d7640 1 -- start start 2023-12-12T09:53:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.457+0000 7f22a54d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a00ce2b0 0x7f22a01e2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.457+0000 7f22a54d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a0143660 0x7f22a00cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.457+0000 7f22a54d7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a0143a40 0x7f22a00ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.457+0000 7f22a54d7640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a009c700 con 0x7f22a0143a40 2023-12-12T09:53:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.457+0000 7f22a54d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a009b850 con 0x7f22a0143660 2023-12-12T09:53:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.457+0000 7f22a54d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a009a7b0 con 0x7f22a00ce2b0 2023-12-12T09:53:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f22a4cd6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a0143a40 0x7f22a00ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f22a4cd6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a0143a40 0x7f22a00ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44794/0 (socket says 172.21.15.72:44794) 2023-12-12T09:53:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f22a4cd6640 1 -- 172.21.15.72:0/3778310404 learned_addr learned my addr 172.21.15.72:0/3778310404 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f229ffff640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a00ce2b0 0x7f22a01e2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f229f7fe640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a0143660 0x7f22a00cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f22a4cd6640 1 -- 172.21.15.72:0/3778310404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a00ce2b0 msgr2=0x7f22a01e2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f22a4cd6640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a00ce2b0 0x7f22a01e2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f22a4cd6640 1 -- 172.21.15.72:0/3778310404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a0143660 msgr2=0x7f22a00cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f22a4cd6640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a0143660 0x7f22a00cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.458+0000 7f22a4cd6640 1 -- 172.21.15.72:0/3778310404 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298009400 con 0x7f22a0143a40 2023-12-12T09:53:25.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.459+0000 7f22a4cd6640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a0143a40 0x7f22a00ccfd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f22980098c0 tx=0x7f22980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.459+0000 7f229cff9640 1 -- 172.21.15.72:0/3778310404 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229803a630 con 0x7f22a0143a40 2023-12-12T09:53:25.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.459+0000 7f229cff9640 1 -- 172.21.15.72:0/3778310404 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229803a790 con 0x7f22a0143a40 2023-12-12T09:53:25.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.459+0000 7f229cff9640 1 -- 172.21.15.72:0/3778310404 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229803ab40 con 0x7f22a0143a40 2023-12-12T09:53:25.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.459+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a009aa90 con 0x7f22a0143a40 2023-12-12T09:53:25.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.459+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a009b050 con 0x7f22a0143a40 2023-12-12T09:53:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.460+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a0067910 con 0x7f22a0143a40 2023-12-12T09:53:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.461+0000 7f229cff9640 1 -- 172.21.15.72:0/3778310404 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f229803aca0 con 0x7f22a0143a40 2023-12-12T09:53:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.462+0000 7f229cff9640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f228006ae40 0x7f228006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.462+0000 7f229cff9640 1 -- 172.21.15.72:0/3778310404 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f22980ab640 con 0x7f22a0143a40 2023-12-12T09:53:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.462+0000 7f229ffff640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f228006ae40 0x7f228006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.463+0000 7f229ffff640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f228006ae40 0x7f228006d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2290001560 tx=0x7f2290009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.466+0000 7f229cff9640 1 -- 172.21.15.72:0/3778310404 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2298076330 con 0x7f22a0143a40 2023-12-12T09:53:25.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.644+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f22a00b3260 con 0x7f22a0143a40 2023-12-12T09:53:25.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.645+0000 7f229cff9640 1 -- 172.21.15.72:0/3778310404 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2298079950 con 0x7f22a0143a40 2023-12-12T09:53:25.647 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:25.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.647+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f228006ae40 msgr2=0x7f228006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.647+0000 7f22a54d7640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f228006ae40 0x7f228006d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2290001560 tx=0x7f2290009040 comp rx=0 tx=0).stop 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.647+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a0143a40 msgr2=0x7f22a00ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.647+0000 7f22a54d7640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a0143a40 0x7f22a00ccfd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f22980098c0 tx=0x7f22980046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.648+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 shutdown_connections 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.648+0000 7f22a54d7640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f228006ae40 0x7f228006d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.648+0000 7f22a54d7640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22a0143a40 0x7f22a00ccfd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.648+0000 7f22a54d7640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a0143660 0x7f22a00cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.648+0000 7f22a54d7640 1 --2- 172.21.15.72:0/3778310404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22a00ce2b0 0x7f22a01e2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.648+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 >> 172.21.15.72:0/3778310404 conn(0x7f22a00b0940 msgr2=0x7f22a00affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.648+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 shutdown_connections 2023-12-12T09:53:25.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.648+0000 7f22a54d7640 1 -- 172.21.15.72:0/3778310404 wait complete. 2023-12-12T09:53:25.667 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:53:25.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.873+0000 7f4f39e35640 1 Processor -- start 2023-12-12T09:53:25.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.874+0000 7f4f39e35640 1 -- start start 2023-12-12T09:53:25.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.875+0000 7f4f39e35640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d13c0 0x7f4f340ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.875+0000 7f4f39e35640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4f340b0760 0x7f4f340b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.875+0000 7f4f39e35640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f340b2280 con 0x7f4f340d0d70 2023-12-12T09:53:25.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.875+0000 7f4f39e35640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f340cee70 con 0x7f4f340b0760 2023-12-12T09:53:25.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.875+0000 7f4f39e35640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f340d1190 con 0x7f4f340d13c0 2023-12-12T09:53:25.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.875+0000 7f4f39634640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4f340b0760 0x7f4f340b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.875+0000 7f4f33fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4f340d0d70 0x7f4f340d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39296/0 (socket says 172.21.15.72:39296) 2023-12-12T09:53:25.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.875+0000 7f4f33fff640 1 -- 172.21.15.72:0/2785064662 learned_addr learned my addr 172.21.15.72:0/2785064662 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:25.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.876+0000 7f4f38e33640 1 --2- 172.21.15.72:0/2785064662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d13c0 0x7f4f340ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.876+0000 7f4f39634640 1 -- 172.21.15.72:0/2785064662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d13c0 msgr2=0x7f4f340ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:25.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.876+0000 7f4f39634640 1 --2- 172.21.15.72:0/2785064662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d13c0 0x7f4f340ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.876+0000 7f4f39634640 1 -- 172.21.15.72:0/2785064662 >> v1:172.21.15.72:6789/0 conn(0x7f4f340d0d70 legacy=0x7f4f340d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.876+0000 7f4f39634640 1 -- 172.21.15.72:0/2785064662 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f340ceb90 con 0x7f4f340b0760 2023-12-12T09:53:25.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.877+0000 7f4f39634640 1 --2- 172.21.15.72:0/2785064662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340b0760 0x7f4f340b0bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0097a0 tx=0x7f4f2c02b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dd0ade30787903 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.877+0000 7f4f337fe640 1 -- 172.21.15.72:0/2785064662 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c030710 con 0x7f4f340b0760 2023-12-12T09:53:25.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.877+0000 7f4f337fe640 1 -- 172.21.15.72:0/2785064662 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c02e050 con 0x7f4f340b0760 2023-12-12T09:53:25.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.877+0000 7f4f337fe640 1 -- 172.21.15.72:0/2785064662 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c030a90 con 0x7f4f340b0760 2023-12-12T09:53:25.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.878+0000 7f4f39e35640 1 -- 172.21.15.72:0/2785064662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340b0760 msgr2=0x7f4f340b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.878+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2785064662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340b0760 0x7f4f340b0bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0097a0 tx=0x7f4f2c02b660 comp rx=0 tx=0).stop 2023-12-12T09:53:25.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.878+0000 7f4f39e35640 1 -- 172.21.15.72:0/2785064662 shutdown_connections 2023-12-12T09:53:25.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.878+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2785064662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340b0760 0x7f4f340b0bf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.878+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2785064662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d13c0 0x7f4f340ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.878+0000 7f4f39e35640 1 -- 172.21.15.72:0/2785064662 >> 172.21.15.72:0/2785064662 conn(0x7f4f3409c260 msgr2=0x7f4f340d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:25.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.878+0000 7f4f39e35640 1 -- 172.21.15.72:0/2785064662 shutdown_connections 2023-12-12T09:53:25.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.878+0000 7f4f39e35640 1 -- 172.21.15.72:0/2785064662 wait complete. 2023-12-12T09:53:25.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.880+0000 7f4f39e35640 1 Processor -- start 2023-12-12T09:53:25.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.880+0000 7f4f39e35640 1 -- start start 2023-12-12T09:53:25.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.881+0000 7f4f39e35640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f340b0760 0x7f4f340c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.881+0000 7f4f39e35640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d0d70 0x7f4f340b8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.881+0000 7f4f39e35640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340d13c0 0x7f4f340b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.881+0000 7f4f39e35640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3415a9d0 con 0x7f4f340d13c0 2023-12-12T09:53:25.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.881+0000 7f4f39e35640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f340c9760 con 0x7f4f340b0760 2023-12-12T09:53:25.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.881+0000 7f4f39e35640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f340b3670 con 0x7f4f340d0d70 2023-12-12T09:53:25.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340d13c0 0x7f4f340b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340d13c0 0x7f4f340b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44812/0 (socket says 172.21.15.72:44812) 2023-12-12T09:53:25.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 -- 172.21.15.72:0/2421410683 learned_addr learned my addr 172.21.15.72:0/2421410683 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f38e33640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f340b0760 0x7f4f340c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f33fff640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d0d70 0x7f4f340b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 -- 172.21.15.72:0/2421410683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d0d70 msgr2=0x7f4f340b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d0d70 0x7f4f340b8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 -- 172.21.15.72:0/2421410683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f340b0760 msgr2=0x7f4f340c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f340b0760 0x7f4f340c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 -- 172.21.15.72:0/2421410683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c009400 con 0x7f4f340d13c0 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.882+0000 7f4f39634640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340d13c0 0x7f4f340b4430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c035040 tx=0x7f4f2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.883+0000 7f4f317fa640 1 -- 172.21.15.72:0/2421410683 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c03b810 con 0x7f4f340d13c0 2023-12-12T09:53:25.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.883+0000 7f4f317fa640 1 -- 172.21.15.72:0/2421410683 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c03b990 con 0x7f4f340d13c0 2023-12-12T09:53:25.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.883+0000 7f4f317fa640 1 -- 172.21.15.72:0/2421410683 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c03bd40 con 0x7f4f340d13c0 2023-12-12T09:53:25.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.883+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f340b3950 con 0x7f4f340d13c0 2023-12-12T09:53:25.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.883+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f340b3f10 con 0x7f4f340d13c0 2023-12-12T09:53:25.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.884+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f34074450 con 0x7f4f340d13c0 2023-12-12T09:53:25.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.885+0000 7f4f317fa640 1 -- 172.21.15.72:0/2421410683 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f2c039040 con 0x7f4f340d13c0 2023-12-12T09:53:25.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.886+0000 7f4f317fa640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4f1406ae40 0x7f4f1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:25.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.886+0000 7f4f317fa640 1 -- 172.21.15.72:0/2421410683 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4f2c0ab050 con 0x7f4f340d13c0 2023-12-12T09:53:25.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.886+0000 7f4f38e33640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4f1406ae40 0x7f4f1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:25.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.887+0000 7f4f38e33640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4f1406ae40 0x7f4f1406d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c006fd0 tx=0x7f4f1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:25.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:25.891+0000 7f4f317fa640 1 -- 172.21.15.72:0/2421410683 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f2c075cb0 con 0x7f4f340d13c0 2023-12-12T09:53:26.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.093+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4f340d2d70 con 0x7f4f340d13c0 2023-12-12T09:53:26.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.093+0000 7f4f317fa640 1 -- 172.21.15.72:0/2421410683 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4f2c037080 con 0x7f4f340d13c0 2023-12-12T09:53:26.095 INFO:teuthology.orchestra.run.smithi072.stderr:listed 0 entries 2023-12-12T09:53:26.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4f1406ae40 msgr2=0x7f4f1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4f1406ae40 0x7f4f1406d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c006fd0 tx=0x7f4f1c008040 comp rx=0 tx=0).stop 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340d13c0 msgr2=0x7f4f340b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340d13c0 0x7f4f340b4430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c035040 tx=0x7f4f2c0046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 shutdown_connections 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4f1406ae40 0x7f4f1406d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f340d13c0 0x7f4f340b4430 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f340d0d70 0x7f4f340b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 --2- 172.21.15.72:0/2421410683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f340b0760 0x7f4f340c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 >> 172.21.15.72:0/2421410683 conn(0x7f4f3409c260 msgr2=0x7f4f340b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 shutdown_connections 2023-12-12T09:53:26.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.096+0000 7f4f39e35640 1 -- 172.21.15.72:0/2421410683 wait complete. 2023-12-12T09:53:26.117 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-12-12T09:53:26.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:26.117 DEBUG:teuthology.orchestra.run.smithi072:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-12-12T09:53:26.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:53:26.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:26.206 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-12T09:53:26.257 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:53:26.257 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:26.319 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:53:26.319 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:26.373 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:53:26.374 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:26.427 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:26.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.648+0000 7fdd2eb8e640 1 Processor -- start 2023-12-12T09:53:26.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.648+0000 7fdd2eb8e640 1 -- start start 2023-12-12T09:53:26.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.650+0000 7fdd2eb8e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.650+0000 7fdd2eb8e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd280b3bb0 con 0x7fdd2806c410 2023-12-12T09:53:26.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.650+0000 7fdd2eb8e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd281441e0 con 0x7fdd28072180 2023-12-12T09:53:26.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.650+0000 7fdd2eb8e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28072680 con 0x7fdd28143e00 2023-12-12T09:53:26.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.650+0000 7fdd2d12b640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fdd2806c410 0x7fdd2815ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39312/0 (socket says 172.21.15.72:39312) 2023-12-12T09:53:26.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.650+0000 7fdd2d12b640 1 -- 172.21.15.72:0/3974692030 learned_addr learned my addr 172.21.15.72:0/3974692030 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:26.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.651+0000 7fdd2c92a640 1 --2- 172.21.15.72:0/3974692030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.651+0000 7fdd277fe640 1 -- 172.21.15.72:0/3974692030 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220197084 0 0) 0x7fdd280b3bb0 con 0x7fdd2806c410 2023-12-12T09:53:26.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.651+0000 7fdd277fe640 1 -- 172.21.15.72:0/3974692030 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd18003520 con 0x7fdd2806c410 2023-12-12T09:53:26.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.652+0000 7fdd2c92a640 1 -- 172.21.15.72:0/3974692030 >> v1:172.21.15.113:6789/0 conn(0x7fdd28072180 legacy=0x7fdd280b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.652+0000 7fdd2c92a640 1 -- 172.21.15.72:0/3974692030 >> v1:172.21.15.72:6789/0 conn(0x7fdd2806c410 legacy=0x7fdd2815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.652+0000 7fdd2c92a640 1 -- 172.21.15.72:0/3974692030 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280b1e30 con 0x7fdd28143e00 2023-12-12T09:53:26.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.653+0000 7fdd2c92a640 1 --2- 172.21.15.72:0/3974692030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280b4040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0097d0 tx=0x7fdd1c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e530cb72383a23e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:26.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.654+0000 7fdd277fe640 1 -- 172.21.15.72:0/3974692030 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c031890 con 0x7fdd28143e00 2023-12-12T09:53:26.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.654+0000 7fdd277fe640 1 -- 172.21.15.72:0/3974692030 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0319f0 con 0x7fdd28143e00 2023-12-12T09:53:26.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.654+0000 7fdd277fe640 1 -- 172.21.15.72:0/3974692030 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c031d80 con 0x7fdd28143e00 2023-12-12T09:53:26.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.655+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/3974692030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 msgr2=0x7fdd280b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.655+0000 7fdd2eb8e640 1 --2- 172.21.15.72:0/3974692030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280b4040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0097d0 tx=0x7fdd1c02b930 comp rx=0 tx=0).stop 2023-12-12T09:53:26.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.655+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/3974692030 shutdown_connections 2023-12-12T09:53:26.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.655+0000 7fdd2eb8e640 1 --2- 172.21.15.72:0/3974692030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280b4040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.655+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/3974692030 >> 172.21.15.72:0/3974692030 conn(0x7fdd280b0a90 msgr2=0x7fdd280b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:26.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.655+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/3974692030 shutdown_connections 2023-12-12T09:53:26.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.655+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/3974692030 wait complete. 2023-12-12T09:53:26.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.657+0000 7fdd2eb8e640 1 Processor -- start 2023-12-12T09:53:26.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.657+0000 7fdd2eb8e640 1 -- start start 2023-12-12T09:53:26.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd2eb8e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd2806c410 0x7fdd28152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd2eb8e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28072180 0x7fdd280adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd2eb8e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd2eb8e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28153160 con 0x7fdd28072180 2023-12-12T09:53:26.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd2eb8e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28138970 con 0x7fdd2806c410 2023-12-12T09:53:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd2eb8e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd280ad610 con 0x7fdd28143e00 2023-12-12T09:53:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28072180 0x7fdd280adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28072180 0x7fdd280adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44822/0 (socket says 172.21.15.72:44822) 2023-12-12T09:53:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.658+0000 7fdd27fff640 1 -- 172.21.15.72:0/1476104469 learned_addr learned my addr 172.21.15.72:0/1476104469 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd2c92a640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd2806c410 0x7fdd28152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd2d12b640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd27fff640 1 -- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 msgr2=0x7fdd280cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd27fff640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd27fff640 1 -- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd2806c410 msgr2=0x7fdd28152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd27fff640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd2806c410 0x7fdd28152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd27fff640 1 -- 172.21.15.72:0/1476104469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c009400 con 0x7fdd28072180 2023-12-12T09:53:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd2d12b640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd27fff640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28072180 0x7fdd280adaa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400c450 tx=0x7fdd1400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd257fa640 1 -- 172.21.15.72:0/1476104469 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd140052d0 con 0x7fdd28072180 2023-12-12T09:53:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd257fa640 1 -- 172.21.15.72:0/1476104469 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd14005430 con 0x7fdd28072180 2023-12-12T09:53:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd28152b50 con 0x7fdd28072180 2023-12-12T09:53:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.659+0000 7fdd257fa640 1 -- 172.21.15.72:0/1476104469 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14012870 con 0x7fdd28072180 2023-12-12T09:53:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.660+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd280acce0 con 0x7fdd28072180 2023-12-12T09:53:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.661+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd280ad780 con 0x7fdd28072180 2023-12-12T09:53:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.662+0000 7fdd257fa640 1 -- 172.21.15.72:0/1476104469 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd140129d0 con 0x7fdd28072180 2023-12-12T09:53:26.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.663+0000 7fdd257fa640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd0c06ae70 0x7fdd0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.663+0000 7fdd257fa640 1 -- 172.21.15.72:0/1476104469 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdd140894d0 con 0x7fdd28072180 2023-12-12T09:53:26.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.663+0000 7fdd2c92a640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd0c06ae70 0x7fdd0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.664+0000 7fdd2c92a640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd0c06ae70 0x7fdd0c06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c036000 tx=0x7fdd1c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:26.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.668+0000 7fdd257fa640 1 -- 172.21.15.72:0/1476104469 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd140541c0 con 0x7fdd28072180 2023-12-12T09:53:26.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.859+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd280745d0 con 0x7fdd28072180 2023-12-12T09:53:26.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.862+0000 7fdd257fa640 1 -- 172.21.15.72:0/1476104469 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7fdd140577e0 con 0x7fdd28072180 2023-12-12T09:53:26.864 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:26.865 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd0c06ae70 msgr2=0x7fdd0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd0c06ae70 0x7fdd0c06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c036000 tx=0x7fdd1c03c040 comp rx=0 tx=0).stop 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28072180 msgr2=0x7fdd280adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28072180 0x7fdd280adaa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400c450 tx=0x7fdd1400c910 comp rx=0 tx=0).stop 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 shutdown_connections 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd0c06ae70 0x7fdd0c06d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd28143e00 0x7fdd280cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28072180 0x7fdd280adaa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 --2- 172.21.15.72:0/1476104469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd2806c410 0x7fdd28152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 >> 172.21.15.72:0/1476104469 conn(0x7fdd280b0a90 msgr2=0x7fdd28144400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:26.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 shutdown_connections 2023-12-12T09:53:26.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:26.866+0000 7fdd2eb8e640 1 -- 172.21.15.72:0/1476104469 wait complete. 2023-12-12T09:53:26.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:26.889 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:27.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.096+0000 7feaa4143640 1 Processor -- start 2023-12-12T09:53:27.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.096+0000 7feaa4143640 1 -- start start 2023-12-12T09:53:27.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.097+0000 7feaa4143640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fea9c15d0e0 0x7fea9c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.098+0000 7feaa4143640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea9c072210 0x7fea9c161530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.098+0000 7feaa4143640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c160dc0 con 0x7fea9c161a70 2023-12-12T09:53:27.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.098+0000 7feaa4143640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c070280 con 0x7fea9c15d0e0 2023-12-12T09:53:27.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.098+0000 7feaa4143640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c070560 con 0x7fea9c072210 2023-12-12T09:53:27.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.098+0000 7feaa1edf640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fea9c15d0e0 0x7fea9c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.098+0000 7feaa26e0640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fea9c161a70 0x7fea9c0c92e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39314/0 (socket says 172.21.15.72:39314) 2023-12-12T09:53:27.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.098+0000 7feaa26e0640 1 -- 172.21.15.72:0/2575231477 learned_addr learned my addr 172.21.15.72:0/2575231477 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:27.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.099+0000 7feaa1edf640 1 -- 172.21.15.72:0/2575231477 >> v2:172.21.15.113:3300/0 conn(0x7fea9c072210 msgr2=0x7fea9c161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:27.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.099+0000 7feaa1edf640 1 --2- 172.21.15.72:0/2575231477 >> v2:172.21.15.113:3300/0 conn(0x7fea9c072210 0x7fea9c161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.099+0000 7feaa1edf640 1 -- 172.21.15.72:0/2575231477 >> v1:172.21.15.72:6789/0 conn(0x7fea9c161a70 legacy=0x7fea9c0c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.099+0000 7feaa1edf640 1 -- 172.21.15.72:0/2575231477 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c070840 con 0x7fea9c15d0e0 2023-12-12T09:53:27.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.099+0000 7feaa0edd640 1 -- 172.21.15.72:0/2575231477 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510548323 0 0) 0x7fea9c160dc0 con 0x7fea9c161a70 2023-12-12T09:53:27.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.100+0000 7feaa1edf640 1 --2- 172.21.15.72:0/2575231477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c15d0e0 0x7fea9c0ce3a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0b1880 tx=0x7fea9802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffb5feb80bff117 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.100+0000 7feaa0edd640 1 -- 172.21.15.72:0/2575231477 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98031830 con 0x7fea9c15d0e0 2023-12-12T09:53:27.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.100+0000 7feaa0edd640 1 -- 172.21.15.72:0/2575231477 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea98031990 con 0x7fea9c15d0e0 2023-12-12T09:53:27.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.100+0000 7feaa0edd640 1 -- 172.21.15.72:0/2575231477 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98031d20 con 0x7fea9c15d0e0 2023-12-12T09:53:27.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.101+0000 7feaa4143640 1 -- 172.21.15.72:0/2575231477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c15d0e0 msgr2=0x7fea9c0ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.101+0000 7feaa4143640 1 --2- 172.21.15.72:0/2575231477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c15d0e0 0x7fea9c0ce3a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0b1880 tx=0x7fea9802b8d0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.101+0000 7feaa4143640 1 -- 172.21.15.72:0/2575231477 shutdown_connections 2023-12-12T09:53:27.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.101+0000 7feaa4143640 1 --2- 172.21.15.72:0/2575231477 >> v2:172.21.15.113:3300/0 conn(0x7fea9c072210 0x7fea9c161530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.101+0000 7feaa4143640 1 --2- 172.21.15.72:0/2575231477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c15d0e0 0x7fea9c0ce3a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.101+0000 7feaa4143640 1 -- 172.21.15.72:0/2575231477 >> 172.21.15.72:0/2575231477 conn(0x7fea9c0cee70 msgr2=0x7fea9c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:27.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.101+0000 7feaa4143640 1 -- 172.21.15.72:0/2575231477 shutdown_connections 2023-12-12T09:53:27.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.101+0000 7feaa4143640 1 -- 172.21.15.72:0/2575231477 wait complete. 2023-12-12T09:53:27.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.103+0000 7feaa4143640 1 Processor -- start 2023-12-12T09:53:27.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.103+0000 7feaa4143640 1 -- start start 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa4143640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea9c072210 0x7fea9c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa4143640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea9c15d0e0 0x7fea9c160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa4143640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c09c610 0x7fea9c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa4143640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c1588c0 con 0x7fea9c09c610 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa4143640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c09c180 con 0x7fea9c15d0e0 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa4143640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c158340 con 0x7fea9c072210 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa26e0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c09c610 0x7fea9c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa26e0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c09c610 0x7fea9c1606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44832/0 (socket says 172.21.15.72:44832) 2023-12-12T09:53:27.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa26e0640 1 -- 172.21.15.72:0/3365281369 learned_addr learned my addr 172.21.15.72:0/3365281369 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:27.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa26e0640 1 -- 172.21.15.72:0/3365281369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea9c072210 msgr2=0x7fea9c157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa26e0640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea9c072210 0x7fea9c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa26e0640 1 -- 172.21.15.72:0/3365281369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea9c15d0e0 msgr2=0x7fea9c160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa26e0640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea9c15d0e0 0x7fea9c160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.104+0000 7feaa26e0640 1 -- 172.21.15.72:0/3365281369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea98009400 con 0x7fea9c09c610 2023-12-12T09:53:27.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.105+0000 7feaa26e0640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c09c610 0x7fea9c1606a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fea9400c780 tx=0x7fea9400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.105+0000 7fea927fc640 1 -- 172.21.15.72:0/3365281369 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9400ce90 con 0x7fea9c09c610 2023-12-12T09:53:27.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.105+0000 7fea927fc640 1 -- 172.21.15.72:0/3365281369 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9400a640 con 0x7fea9c09c610 2023-12-12T09:53:27.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.105+0000 7fea927fc640 1 -- 172.21.15.72:0/3365281369 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9400a9d0 con 0x7fea9c09c610 2023-12-12T09:53:27.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.105+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c09b570 con 0x7fea9c09c610 2023-12-12T09:53:27.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.105+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c09bb30 con 0x7fea9c09c610 2023-12-12T09:53:27.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.106+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c074450 con 0x7fea9c09c610 2023-12-12T09:53:27.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.107+0000 7fea927fc640 1 -- 172.21.15.72:0/3365281369 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea9400ab30 con 0x7fea9c09c610 2023-12-12T09:53:27.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.108+0000 7fea927fc640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.108+0000 7fea927fc640 1 -- 172.21.15.72:0/3365281369 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fea94088e30 con 0x7fea9c09c610 2023-12-12T09:53:27.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.109+0000 7feaa1edf640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.109+0000 7feaa1edf640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea8006ae70 0x7fea8006d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea98036000 tx=0x7fea98030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.113+0000 7fea927fc640 1 -- 172.21.15.72:0/3365281369 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea94053b10 con 0x7fea9c09c610 2023-12-12T09:53:27.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.303+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea9c09c2f0 con 0x7fea9c09c610 2023-12-12T09:53:27.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.304+0000 7fea927fc640 1 -- 172.21.15.72:0/3365281369 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea94018020 con 0x7fea9c09c610 2023-12-12T09:53:27.305 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.306+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea8006ae70 msgr2=0x7fea8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.306+0000 7feaa4143640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea8006ae70 0x7fea8006d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea98036000 tx=0x7fea98030000 comp rx=0 tx=0).stop 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.306+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c09c610 msgr2=0x7fea9c1606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.306+0000 7feaa4143640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c09c610 0x7fea9c1606a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fea9400c780 tx=0x7fea9400cc40 comp rx=0 tx=0).stop 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.307+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 shutdown_connections 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.307+0000 7feaa4143640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.307+0000 7feaa4143640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9c09c610 0x7fea9c1606a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.307+0000 7feaa4143640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea9c15d0e0 0x7fea9c160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.307+0000 7feaa4143640 1 --2- 172.21.15.72:0/3365281369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea9c072210 0x7fea9c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.307+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 >> 172.21.15.72:0/3365281369 conn(0x7fea9c0cee70 msgr2=0x7fea9c15b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:27.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.307+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 shutdown_connections 2023-12-12T09:53:27.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.307+0000 7feaa4143640 1 -- 172.21.15.72:0/3365281369 wait complete. 2023-12-12T09:53:27.325 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:27.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.542+0000 7fe28efa5640 1 Processor -- start 2023-12-12T09:53:27.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.543+0000 7fe28efa5640 1 -- start start 2023-12-12T09:53:27.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.544+0000 7fe28efa5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe28815d5f0 0x7fe2880d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.544+0000 7fe28efa5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d3a50 0x7fe2880cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.544+0000 7fe28efa5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2880d0480 con 0x7fe2880d0030 2023-12-12T09:53:27.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.544+0000 7fe28efa5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28815e620 con 0x7fe28815d5f0 2023-12-12T09:53:27.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.544+0000 7fe28efa5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28815ea40 con 0x7fe2880d3a50 2023-12-12T09:53:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.545+0000 7fe28cd41640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe28815d5f0 0x7fe2880d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.545+0000 7fe287fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d3a50 0x7fe2880cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.545+0000 7fe28cd41640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe28815d5f0 0x7fe2880d1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:60318/0 (socket says 172.21.15.72:60318) 2023-12-12T09:53:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.545+0000 7fe28cd41640 1 -- 172.21.15.72:0/2722430799 learned_addr learned my addr 172.21.15.72:0/2722430799 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.546+0000 7fe28cd41640 1 -- 172.21.15.72:0/2722430799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d3a50 msgr2=0x7fe2880cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.546+0000 7fe28cd41640 1 --2- 172.21.15.72:0/2722430799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d3a50 0x7fe2880cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.546+0000 7fe28cd41640 1 -- 172.21.15.72:0/2722430799 >> v1:172.21.15.113:6789/0 conn(0x7fe2880d0030 legacy=0x7fe288161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.546+0000 7fe28cd41640 1 -- 172.21.15.72:0/2722430799 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880b2300 con 0x7fe28815d5f0 2023-12-12T09:53:27.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.547+0000 7fe28cd41640 1 --2- 172.21.15.72:0/2722430799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28815d5f0 0x7fe2880d1d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0098a0 tx=0x7fe27c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=583d92b703e16f02 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.547+0000 7fe2877fe640 1 -- 172.21.15.72:0/2722430799 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c031640 con 0x7fe28815d5f0 2023-12-12T09:53:27.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.548+0000 7fe2877fe640 1 -- 172.21.15.72:0/2722430799 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c0317a0 con 0x7fe28815d5f0 2023-12-12T09:53:27.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.548+0000 7fe2877fe640 1 -- 172.21.15.72:0/2722430799 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c031b30 con 0x7fe28815d5f0 2023-12-12T09:53:27.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.548+0000 7fe28efa5640 1 -- 172.21.15.72:0/2722430799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28815d5f0 msgr2=0x7fe2880d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.548+0000 7fe28efa5640 1 --2- 172.21.15.72:0/2722430799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28815d5f0 0x7fe2880d1d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0098a0 tx=0x7fe27c02b8f0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.548+0000 7fe28efa5640 1 -- 172.21.15.72:0/2722430799 shutdown_connections 2023-12-12T09:53:27.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.548+0000 7fe28efa5640 1 --2- 172.21.15.72:0/2722430799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d3a50 0x7fe2880cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.548+0000 7fe28efa5640 1 --2- 172.21.15.72:0/2722430799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28815d5f0 0x7fe2880d1d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.548+0000 7fe28efa5640 1 -- 172.21.15.72:0/2722430799 >> 172.21.15.72:0/2722430799 conn(0x7fe28809c450 msgr2=0x7fe2880d3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:27.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.549+0000 7fe28efa5640 1 -- 172.21.15.72:0/2722430799 shutdown_connections 2023-12-12T09:53:27.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.549+0000 7fe28efa5640 1 -- 172.21.15.72:0/2722430799 wait complete. 2023-12-12T09:53:27.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.550+0000 7fe28efa5640 1 Processor -- start 2023-12-12T09:53:27.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.551+0000 7fe28efa5640 1 -- start start 2023-12-12T09:53:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.551+0000 7fe28efa5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d0030 0x7fe28814b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28efa5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2880d3a50 0x7fe288159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28efa5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe28815d5f0 0x7fe2880b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28efa5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28815a720 con 0x7fe28815d5f0 2023-12-12T09:53:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28efa5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28814bbf0 con 0x7fe2880d3a50 2023-12-12T09:53:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28efa5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe288159aa0 con 0x7fe2880d0030 2023-12-12T09:53:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28d542640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe28815d5f0 0x7fe2880b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28d542640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe28815d5f0 0x7fe2880b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44840/0 (socket says 172.21.15.72:44840) 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28d542640 1 -- 172.21.15.72:0/1643385012 learned_addr learned my addr 172.21.15.72:0/1643385012 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe287fff640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2880d3a50 0x7fe288159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28cd41640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d0030 0x7fe28814b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28d542640 1 -- 172.21.15.72:0/1643385012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d0030 msgr2=0x7fe28814b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28d542640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d0030 0x7fe28814b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28d542640 1 -- 172.21.15.72:0/1643385012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2880d3a50 msgr2=0x7fe288159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28d542640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2880d3a50 0x7fe288159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.552+0000 7fe28d542640 1 -- 172.21.15.72:0/1643385012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c0094d0 con 0x7fe28815d5f0 2023-12-12T09:53:27.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.553+0000 7fe28d542640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe28815d5f0 0x7fe2880b2a60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe28000c7a0 tx=0x7fe28000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.553+0000 7fe2857fa640 1 -- 172.21.15.72:0/1643385012 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280003b70 con 0x7fe28815d5f0 2023-12-12T09:53:27.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.553+0000 7fe2857fa640 1 -- 172.21.15.72:0/1643385012 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe280003cf0 con 0x7fe28815d5f0 2023-12-12T09:53:27.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.553+0000 7fe2857fa640 1 -- 172.21.15.72:0/1643385012 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280003ea0 con 0x7fe28815d5f0 2023-12-12T09:53:27.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.553+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe288159d80 con 0x7fe28815d5f0 2023-12-12T09:53:27.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.553+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28815a340 con 0x7fe28815d5f0 2023-12-12T09:53:27.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.554+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2880745d0 con 0x7fe28815d5f0 2023-12-12T09:53:27.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.555+0000 7fe2857fa640 1 -- 172.21.15.72:0/1643385012 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe28000aa90 con 0x7fe28815d5f0 2023-12-12T09:53:27.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.556+0000 7fe2857fa640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe26c06ae40 0x7fe26c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.556+0000 7fe2857fa640 1 -- 172.21.15.72:0/1643385012 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe280089720 con 0x7fe28815d5f0 2023-12-12T09:53:27.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.556+0000 7fe28cd41640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe26c06ae40 0x7fe26c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.557+0000 7fe28cd41640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe26c06ae40 0x7fe26c06d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe27c036000 tx=0x7fe27c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.562+0000 7fe2857fa640 1 -- 172.21.15.72:0/1643385012 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe280054370 con 0x7fe28815d5f0 2023-12-12T09:53:27.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.747+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2880d0480 con 0x7fe28815d5f0 2023-12-12T09:53:27.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.748+0000 7fe2857fa640 1 -- 172.21.15.72:0/1643385012 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2880d0480 con 0x7fe28815d5f0 2023-12-12T09:53:27.750 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:27.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe26c06ae40 msgr2=0x7fe26c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe26c06ae40 0x7fe26c06d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe27c036000 tx=0x7fe27c03c040 comp rx=0 tx=0).stop 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe28815d5f0 msgr2=0x7fe2880b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe28815d5f0 0x7fe2880b2a60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe28000c7a0 tx=0x7fe28000cc60 comp rx=0 tx=0).stop 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 shutdown_connections 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe26c06ae40 0x7fe26c06d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe28815d5f0 0x7fe2880b2a60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2880d3a50 0x7fe288159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 --2- 172.21.15.72:0/1643385012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2880d0030 0x7fe28814b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 >> 172.21.15.72:0/1643385012 conn(0x7fe28809c450 msgr2=0x7fe2880b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 shutdown_connections 2023-12-12T09:53:27.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.750+0000 7fe28efa5640 1 -- 172.21.15.72:0/1643385012 wait complete. 2023-12-12T09:53:27.768 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:27.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.972+0000 7f681004f640 1 Processor -- start 2023-12-12T09:53:27.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.973+0000 7f681004f640 1 -- start start 2023-12-12T09:53:27.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.974+0000 7f681004f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f68080cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.974+0000 7f681004f640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f68080d1a00 0x7f6808099dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.974+0000 7f681004f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f680806e4d0 0x7f68080cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.974+0000 7f681004f640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68080d04f0 con 0x7f68080d1a00 2023-12-12T09:53:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.974+0000 7f681004f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680809a310 con 0x7f680806e4d0 2023-12-12T09:53:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.974+0000 7f681004f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68080cf1f0 con 0x7f68080999f0 2023-12-12T09:53:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680d5ea640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f68080d1a00 0x7f6808099dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680d5ea640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f68080d1a00 0x7f6808099dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44844/0 (socket says 172.21.15.72:44844) 2023-12-12T09:53:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680d5ea640 1 -- 172.21.15.72:0/2644771106 learned_addr learned my addr 172.21.15.72:0/2644771106 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680e5ec640 1 --2- 172.21.15.72:0/2644771106 >> v2:172.21.15.113:3300/0 conn(0x7f680806e4d0 0x7f68080cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680ddeb640 1 --2- 172.21.15.72:0/2644771106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f68080cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680d5ea640 1 -- 172.21.15.72:0/2644771106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 msgr2=0x7f68080cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680d5ea640 1 --2- 172.21.15.72:0/2644771106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f68080cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680d5ea640 1 -- 172.21.15.72:0/2644771106 >> v2:172.21.15.113:3300/0 conn(0x7f680806e4d0 msgr2=0x7f68080cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680d5ea640 1 --2- 172.21.15.72:0/2644771106 >> v2:172.21.15.113:3300/0 conn(0x7f680806e4d0 0x7f68080cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680d5ea640 1 -- 172.21.15.72:0/2644771106 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080cf4d0 con 0x7f68080d1a00 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.975+0000 7f680ddeb640 1 --2- 172.21.15.72:0/2644771106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f68080cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.976+0000 7f680d5ea640 1 --2- 172.21.15.72:0/2644771106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808099dd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6808071f10 tx=0x7f67f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd73fe893ecb11d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.977+0000 7f680cde9640 1 -- 172.21.15.72:0/2644771106 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f40317e0 con 0x7f68080d1a00 2023-12-12T09:53:27.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.977+0000 7f680cde9640 1 -- 172.21.15.72:0/2644771106 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f4031940 con 0x7f68080d1a00 2023-12-12T09:53:27.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.977+0000 7f680cde9640 1 -- 172.21.15.72:0/2644771106 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4031cd0 con 0x7f68080d1a00 2023-12-12T09:53:27.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.977+0000 7f681004f640 1 -- 172.21.15.72:0/2644771106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 msgr2=0x7f6808099dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.977+0000 7f681004f640 1 --2- 172.21.15.72:0/2644771106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808099dd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6808071f10 tx=0x7f67f402b880 comp rx=0 tx=0).stop 2023-12-12T09:53:27.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.978+0000 7f681004f640 1 -- 172.21.15.72:0/2644771106 shutdown_connections 2023-12-12T09:53:27.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.978+0000 7f681004f640 1 --2- 172.21.15.72:0/2644771106 >> v2:172.21.15.113:3300/0 conn(0x7f680806e4d0 0x7f68080cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.978+0000 7f681004f640 1 --2- 172.21.15.72:0/2644771106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808099dd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.978+0000 7f681004f640 1 --2- 172.21.15.72:0/2644771106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f68080cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.978+0000 7f681004f640 1 -- 172.21.15.72:0/2644771106 >> 172.21.15.72:0/2644771106 conn(0x7f68080d0ee0 msgr2=0x7f68080d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:27.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.978+0000 7f681004f640 1 -- 172.21.15.72:0/2644771106 shutdown_connections 2023-12-12T09:53:27.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.978+0000 7f681004f640 1 -- 172.21.15.72:0/2644771106 wait complete. 2023-12-12T09:53:27.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.979+0000 7f681004f640 1 Processor -- start 2023-12-12T09:53:27.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.980+0000 7f681004f640 1 -- start start 2023-12-12T09:53:27.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.980+0000 7f681004f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f680806e4d0 0x7f680815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.980+0000 7f681004f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f6808152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f681004f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f681004f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680815b060 con 0x7f68080d1a00 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f681004f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081e6d30 con 0x7f680806e4d0 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f681004f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68081f69a0 con 0x7f68080999f0 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680e5ec640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680e5ec640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44858/0 (socket says 172.21.15.72:44858) 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680e5ec640 1 -- 172.21.15.72:0/2432767522 learned_addr learned my addr 172.21.15.72:0/2432767522 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680ddeb640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f680806e4d0 0x7f680815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680d5ea640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f6808152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680e5ec640 1 -- 172.21.15.72:0/2432767522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 msgr2=0x7f6808152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680e5ec640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f6808152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680e5ec640 1 -- 172.21.15.72:0/2432767522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f680806e4d0 msgr2=0x7f680815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680e5ec640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f680806e4d0 0x7f680815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.981+0000 7f680e5ec640 1 -- 172.21.15.72:0/2432767522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f40094d0 con 0x7f68080d1a00 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.982+0000 7f680e5ec640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808157d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f680000d600 tx=0x7f680000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.982+0000 7f68067fc640 1 -- 172.21.15.72:0/2432767522 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800014620 con 0x7f68080d1a00 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.982+0000 7f68067fc640 1 -- 172.21.15.72:0/2432767522 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68000147a0 con 0x7f68080d1a00 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.982+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680815acc0 con 0x7f68080d1a00 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.982+0000 7f68067fc640 1 -- 172.21.15.72:0/2432767522 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800014b50 con 0x7f68080d1a00 2023-12-12T09:53:27.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.982+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68081585c0 con 0x7f68080d1a00 2023-12-12T09:53:27.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.983+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680815b1a0 con 0x7f68080d1a00 2023-12-12T09:53:27.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.984+0000 7f68067fc640 1 -- 172.21.15.72:0/2432767522 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6800014cb0 con 0x7f68080d1a00 2023-12-12T09:53:27.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.985+0000 7f68067fc640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67ec06ae70 0x7f67ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:27.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.985+0000 7f68067fc640 1 -- 172.21.15.72:0/2432767522 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6800089cd0 con 0x7f68080d1a00 2023-12-12T09:53:27.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.985+0000 7f680ddeb640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67ec06ae70 0x7f67ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:27.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.986+0000 7f680ddeb640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67ec06ae70 0x7f67ec06d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f67fc007510 tx=0x7f67fc0074a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:27.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:27.989+0000 7f68067fc640 1 -- 172.21.15.72:0/2432767522 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f680815b1a0 con 0x7f68080d1a00 2023-12-12T09:53:28.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.180+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f680815b1a0 con 0x7f68080d1a00 2023-12-12T09:53:28.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.181+0000 7f68067fc640 1 -- 172.21.15.72:0/2432767522 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f680815b1a0 con 0x7f68080d1a00 2023-12-12T09:53:28.184 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:28.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67ec06ae70 msgr2=0x7f67ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67ec06ae70 0x7f67ec06d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f67fc007510 tx=0x7f67fc0074a0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 msgr2=0x7f6808157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808157d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f680000d600 tx=0x7f680000dac0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 shutdown_connections 2023-12-12T09:53:28.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67ec06ae70 0x7f67ec06d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68080d1a00 0x7f6808157d70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68080999f0 0x7f6808152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 --2- 172.21.15.72:0/2432767522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f680806e4d0 0x7f680815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.184+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 >> 172.21.15.72:0/2432767522 conn(0x7f68080d0ee0 msgr2=0x7f6808161750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:28.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.185+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 shutdown_connections 2023-12-12T09:53:28.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.185+0000 7f681004f640 1 -- 172.21.15.72:0/2432767522 wait complete. 2023-12-12T09:53:28.203 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:28.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.404+0000 7f501a3cf640 1 Processor -- start 2023-12-12T09:53:28.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.405+0000 7f501a3cf640 1 -- start start 2023-12-12T09:53:28.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.406+0000 7f501a3cf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f501415ab60 0x7f50140cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.406+0000 7f501a3cf640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50140cb3a0 con 0x7f5014160c60 2023-12-12T09:53:28.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.406+0000 7f501a3cf640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50140cb680 con 0x7f50140b3ca0 2023-12-12T09:53:28.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.406+0000 7f501a3cf640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50140cb960 con 0x7f501415ab60 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.406+0000 7f5018bcc640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f5014160c60 0x7f50140b3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39320/0 (socket says 172.21.15.72:39320) 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.406+0000 7f5018bcc640 1 -- 172.21.15.72:0/795789735 learned_addr learned my addr 172.21.15.72:0/795789735 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f50193cd640 1 --2- 172.21.15.72:0/795789735 >> v2:172.21.15.113:3300/0 conn(0x7f501415ab60 0x7f50140cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658952537 0 0) 0x7f50140cb3a0 con 0x7f5014160c60 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5000003520 con 0x7f5014160c60 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1102574306 0 0) 0x7f5000003520 con 0x7f5014160c60 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 >> v2:172.21.15.113:3300/0 conn(0x7f501415ab60 msgr2=0x7f50140cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 --2- 172.21.15.72:0/795789735 >> v2:172.21.15.113:3300/0 conn(0x7f501415ab60 0x7f50140cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 >> v1:172.21.15.113:6789/0 conn(0x7f50140b3ca0 legacy=0x7f501406c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140cbc40 con 0x7f5014160c60 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f5004001670 con 0x7f5014160c60 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 >> v1:172.21.15.72:6789/0 conn(0x7f5014160c60 legacy=0x7f50140b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50000039c0 0x7f5000003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50000047c0 0x7f5000004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5000005180 0x7f500000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140cb960 con 0x7f5000005180 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500000c080 con 0x7f50000039c0 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.407+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500000c240 con 0x7f50000047c0 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.408+0000 7f5019bce640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5000005180 0x7f500000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.408+0000 7f50193cd640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50000039c0 0x7f5000003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.408+0000 7f5018bcc640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50000047c0 0x7f5000004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.408+0000 7f5019bce640 1 -- 172.21.15.72:0/795789735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50000047c0 msgr2=0x7f5000004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.408+0000 7f5019bce640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50000047c0 0x7f5000004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.408+0000 7f5019bce640 1 -- 172.21.15.72:0/795789735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50000039c0 msgr2=0x7f5000003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.408+0000 7f5019bce640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50000039c0 0x7f5000003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.408+0000 7f5019bce640 1 -- 172.21.15.72:0/795789735 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000003520 con 0x7f5000005180 2023-12-12T09:53:28.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.409+0000 7f5019bce640 1 -- 172.21.15.72:0/795789735 reap_dead start 2023-12-12T09:53:28.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.409+0000 7f5019bce640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5000005180 0x7f500000b980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f500c001e20 tx=0x7f50000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7f2714e3c7802 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.409+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c003b70 con 0x7f5000005180 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.409+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c003cd0 con 0x7f5000005180 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.409+0000 7f500bfff640 1 -- 172.21.15.72:0/795789735 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c005310 con 0x7f5000005180 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.410+0000 7f501a3cf640 1 -- 172.21.15.72:0/795789735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5000005180 msgr2=0x7f500000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.410+0000 7f501a3cf640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5000005180 0x7f500000b980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f500c001e20 tx=0x7f50000057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.410+0000 7f501a3cf640 1 -- 172.21.15.72:0/795789735 shutdown_connections 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.410+0000 7f501a3cf640 1 --2- 172.21.15.72:0/795789735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5000005180 0x7f500000b980 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.410+0000 7f501a3cf640 1 -- 172.21.15.72:0/795789735 >> 172.21.15.72:0/795789735 conn(0x7f50140b0940 msgr2=0x7f50140b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.410+0000 7f501a3cf640 1 -- 172.21.15.72:0/795789735 shutdown_connections 2023-12-12T09:53:28.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.410+0000 7f501a3cf640 1 -- 172.21.15.72:0/795789735 wait complete. 2023-12-12T09:53:28.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.412+0000 7f501a3cf640 1 Processor -- start 2023-12-12T09:53:28.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.412+0000 7f501a3cf640 1 -- start start 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f501a3cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5000005180 0x7f501414b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f501a3cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501415c010 0x7f50140c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f501a3cf640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50140afb30 0x7f50140affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f501a3cf640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140cae10 con 0x7f50140afb30 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f501a3cf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140b0500 con 0x7f501415c010 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f501a3cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50140ca210 con 0x7f5000005180 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f5019bce640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50140afb30 0x7f50140affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f5019bce640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50140afb30 0x7f50140affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44886/0 (socket says 172.21.15.72:44886) 2023-12-12T09:53:28.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.413+0000 7f5019bce640 1 -- 172.21.15.72:0/1246706985 learned_addr learned my addr 172.21.15.72:0/1246706985 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:28.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.414+0000 7f5018bcc640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501415c010 0x7f50140c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.414+0000 7f50193cd640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5000005180 0x7f501414b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.414+0000 7f5019bce640 1 -- 172.21.15.72:0/1246706985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5000005180 msgr2=0x7f501414b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.414+0000 7f5019bce640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5000005180 0x7f501414b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.414+0000 7f5019bce640 1 -- 172.21.15.72:0/1246706985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501415c010 msgr2=0x7f50140c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.414+0000 7f5019bce640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501415c010 0x7f50140c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.414+0000 7f5019bce640 1 -- 172.21.15.72:0/1246706985 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140c9690 con 0x7f50140afb30 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.414+0000 7f5019bce640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50140afb30 0x7f50140affc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f500c001f40 tx=0x7f500c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.415+0000 7f5009ffb640 1 -- 172.21.15.72:0/1246706985 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c006d50 con 0x7f50140afb30 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.415+0000 7f5009ffb640 1 -- 172.21.15.72:0/1246706985 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c007030 con 0x7f50140afb30 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.415+0000 7f5009ffb640 1 -- 172.21.15.72:0/1246706985 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c0073c0 con 0x7f50140afb30 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.415+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50140c9970 con 0x7f50140afb30 2023-12-12T09:53:28.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.415+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50140c9f30 con 0x7f50140afb30 2023-12-12T09:53:28.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.416+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5014067910 con 0x7f50140afb30 2023-12-12T09:53:28.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.417+0000 7f5009ffb640 1 -- 172.21.15.72:0/1246706985 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f500c007520 con 0x7f50140afb30 2023-12-12T09:53:28.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.418+0000 7f5009ffb640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff806ae40 0x7f4ff806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.418+0000 7f5009ffb640 1 -- 172.21.15.72:0/1246706985 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f500c0ab4e0 con 0x7f50140afb30 2023-12-12T09:53:28.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.418+0000 7f50193cd640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff806ae40 0x7f4ff806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.418+0000 7f50193cd640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff806ae40 0x7f4ff806d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5014072420 tx=0x7f5010008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:28.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.423+0000 7f5009ffb640 1 -- 172.21.15.72:0/1246706985 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f500c076340 con 0x7f50140afb30 2023-12-12T09:53:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.600+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5014070070 con 0x7f50140afb30 2023-12-12T09:53:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.601+0000 7f5009ffb640 1 -- 172.21.15.72:0/1246706985 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5014070070 con 0x7f50140afb30 2023-12-12T09:53:28.603 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.603+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff806ae40 msgr2=0x7f4ff806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.603+0000 7f501a3cf640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff806ae40 0x7f4ff806d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5014072420 tx=0x7f5010008040 comp rx=0 tx=0).stop 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.603+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50140afb30 msgr2=0x7f50140affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.603+0000 7f501a3cf640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50140afb30 0x7f50140affc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f500c001f40 tx=0x7f500c0014a0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.604+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 shutdown_connections 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.604+0000 7f501a3cf640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff806ae40 0x7f4ff806d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.604+0000 7f501a3cf640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50140afb30 0x7f50140affc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.604+0000 7f501a3cf640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501415c010 0x7f50140c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.604+0000 7f501a3cf640 1 --2- 172.21.15.72:0/1246706985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5000005180 0x7f501414b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.604+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 >> 172.21.15.72:0/1246706985 conn(0x7f50140b0940 msgr2=0x7f50140b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:28.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.604+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 shutdown_connections 2023-12-12T09:53:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.604+0000 7f501a3cf640 1 -- 172.21.15.72:0/1246706985 wait complete. 2023-12-12T09:53:28.624 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:53:28.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.823+0000 7f663c705640 1 Processor -- start 2023-12-12T09:53:28.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.823+0000 7f663c705640 1 -- start start 2023-12-12T09:53:28.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.824+0000 7f663c705640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f663415e2b0 0x7f6634164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.824+0000 7f663c705640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663406ca40 con 0x7f6634157a60 2023-12-12T09:53:28.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.825+0000 7f663c705640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6634071cd0 con 0x7f663406c410 2023-12-12T09:53:28.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.825+0000 7f663c705640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6634070f10 con 0x7f663415e2b0 2023-12-12T09:53:28.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.825+0000 7f663a4a1640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6634157a60 0x7f663406de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39328/0 (socket says 172.21.15.72:39328) 2023-12-12T09:53:28.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.825+0000 7f663a4a1640 1 -- 172.21.15.72:0/1893706505 learned_addr learned my addr 172.21.15.72:0/1893706505 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.825+0000 7f663aca2640 1 --2- 172.21.15.72:0/1893706505 >> v2:172.21.15.113:3300/0 conn(0x7f663415e2b0 0x7f6634164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.825+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723648941 0 0) 0x7f663406ca40 con 0x7f6634157a60 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.825+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6620003520 con 0x7f6634157a60 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038894923 0 0) 0x7f6620003520 con 0x7f6634157a60 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 >> v2:172.21.15.113:3300/0 conn(0x7f663415e2b0 msgr2=0x7f6634164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 --2- 172.21.15.72:0/1893706505 >> v2:172.21.15.113:3300/0 conn(0x7f663415e2b0 0x7f6634164d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 >> v1:172.21.15.113:6789/0 conn(0x7f663406c410 legacy=0x7f663415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340711f0 con 0x7f6634157a60 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f6630001670 con 0x7f6634157a60 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 >> v1:172.21.15.72:6789/0 conn(0x7f6634157a60 legacy=0x7f663406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66200039c0 0x7f6620003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66200047c0 0x7f6620004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f662000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6634070f10 con 0x7f66200051d0 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662000c120 con 0x7f66200047c0 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662000c2e0 con 0x7f66200039c0 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663aca2640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f662000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f6639ca0640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66200047c0 0x7f6620004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.826+0000 7f663a4a1640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66200039c0 0x7f6620003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.827+0000 7f663aca2640 1 -- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66200039c0 msgr2=0x7f6620003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.827+0000 7f663aca2640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66200039c0 0x7f6620003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.827+0000 7f663aca2640 1 -- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66200047c0 msgr2=0x7f6620004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.827+0000 7f663aca2640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66200047c0 0x7f6620004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.827+0000 7f663aca2640 1 -- 172.21.15.72:0/1893706505 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620003520 con 0x7f66200051d0 2023-12-12T09:53:28.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.827+0000 7f663a4a1640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66200039c0 0x7f6620003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:28.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.827+0000 7f663aca2640 1 -- 172.21.15.72:0/1893706505 reap_dead start 2023-12-12T09:53:28.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.828+0000 7f663aca2640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f662000b980 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f662c009f90 tx=0x7f66200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a1a9baa29930c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:28.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.828+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c02cc20 con 0x7f66200051d0 2023-12-12T09:53:28.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.828+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00f040 con 0x7f66200051d0 2023-12-12T09:53:28.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.828+0000 7f663949f640 1 -- 172.21.15.72:0/1893706505 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c003990 con 0x7f66200051d0 2023-12-12T09:53:28.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.828+0000 7f663c705640 1 -- 172.21.15.72:0/1893706505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 msgr2=0x7f662000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.828+0000 7f663c705640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f662000b980 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f662c009f90 tx=0x7f66200057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.829+0000 7f663c705640 1 -- 172.21.15.72:0/1893706505 shutdown_connections 2023-12-12T09:53:28.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.829+0000 7f663c705640 1 --2- 172.21.15.72:0/1893706505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f662000b980 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.829+0000 7f663c705640 1 -- 172.21.15.72:0/1893706505 >> 172.21.15.72:0/1893706505 conn(0x7f66340b0a90 msgr2=0x7f66340b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:28.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.829+0000 7f663c705640 1 -- 172.21.15.72:0/1893706505 shutdown_connections 2023-12-12T09:53:28.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.829+0000 7f663c705640 1 -- 172.21.15.72:0/1893706505 wait complete. 2023-12-12T09:53:28.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.830+0000 7f663c705640 1 Processor -- start 2023-12-12T09:53:28.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.831+0000 7f663c705640 1 -- start start 2023-12-12T09:53:28.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.831+0000 7f663c705640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f663414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.831+0000 7f663c705640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6634155050 0x7f66340ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.831+0000 7f663c705640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66341554d0 0x7f66340aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.831+0000 7f663c705640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6634158550 con 0x7f66200051d0 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.831+0000 7f663c705640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340ad2e0 con 0x7f6634155050 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.831+0000 7f663c705640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66340ad5c0 con 0x7f66341554d0 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f663414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f663414b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44910/0 (socket says 172.21.15.72:44910) 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 -- 172.21.15.72:0/2882233714 learned_addr learned my addr 172.21.15.72:0/2882233714 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 -- 172.21.15.72:0/2882233714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66341554d0 msgr2=0x7f66340aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66341554d0 0x7f66340aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 -- 172.21.15.72:0/2882233714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6634155050 msgr2=0x7f66340ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6634155050 0x7f66340ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 -- 172.21.15.72:0/2882233714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66341581e0 con 0x7f66200051d0 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f663a4a1640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f663414b9a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f663000d750 tx=0x7f663000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.832+0000 7f662affd640 1 -- 172.21.15.72:0/2882233714 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300074e0 con 0x7f66200051d0 2023-12-12T09:53:28.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.833+0000 7f662affd640 1 -- 172.21.15.72:0/2882233714 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6630007640 con 0x7f66200051d0 2023-12-12T09:53:28.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.833+0000 7f662affd640 1 -- 172.21.15.72:0/2882233714 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300049e0 con 0x7f66200051d0 2023-12-12T09:53:28.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.833+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66340ad8a0 con 0x7f66200051d0 2023-12-12T09:53:28.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.833+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66340acda0 con 0x7f66200051d0 2023-12-12T09:53:28.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.834+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66340745d0 con 0x7f66200051d0 2023-12-12T09:53:28.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.835+0000 7f662affd640 1 -- 172.21.15.72:0/2882233714 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6630004b40 con 0x7f66200051d0 2023-12-12T09:53:28.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.835+0000 7f662affd640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f661806ae40 0x7f661806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:28.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.836+0000 7f662affd640 1 -- 172.21.15.72:0/2882233714 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6630089490 con 0x7f66200051d0 2023-12-12T09:53:28.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.836+0000 7f6639ca0640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f661806ae40 0x7f661806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:28.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.836+0000 7f6639ca0640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f661806ae40 0x7f661806d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f66340b4c00 tx=0x7f6624009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:28.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:28.840+0000 7f662affd640 1 -- 172.21.15.72:0/2882233714 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6630054050 con 0x7f66200051d0 2023-12-12T09:53:29.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.040+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f66340ad020 con 0x7f66200051d0 2023-12-12T09:53:29.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.041+0000 7f662affd640 1 -- 172.21.15.72:0/2882233714 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6630019080 con 0x7f66200051d0 2023-12-12T09:53:29.043 INFO:teuthology.orchestra.run.smithi072.stderr:listed 0 entries 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f661806ae40 msgr2=0x7f661806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f661806ae40 0x7f661806d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f66340b4c00 tx=0x7f6624009040 comp rx=0 tx=0).stop 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 msgr2=0x7f663414b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f663414b9a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f663000d750 tx=0x7f663000dc10 comp rx=0 tx=0).stop 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 shutdown_connections 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f661806ae40 0x7f661806d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66341554d0 0x7f66340aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6634155050 0x7f66340ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 --2- 172.21.15.72:0/2882233714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66200051d0 0x7f663414b9a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 >> 172.21.15.72:0/2882233714 conn(0x7f66340b0a90 msgr2=0x7f66340b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:29.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 shutdown_connections 2023-12-12T09:53:29.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.044+0000 7f663c705640 1 -- 172.21.15.72:0/2882233714 wait complete. 2023-12-12T09:53:29.065 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-12-12T09:53:29.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:29.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-12-12T09:53:29.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:53:29.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:29.174 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-12T09:53:29.224 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-12T09:53:29.224 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:53:29.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.443+0000 7f27f761e640 1 Processor -- start 2023-12-12T09:53:29.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.444+0000 7f27f761e640 1 -- start start 2023-12-12T09:53:29.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.445+0000 7f27f761e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27f006c410 0x7f27f015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.445+0000 7f27f761e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f006ca40 con 0x7f27f015e0e0 2023-12-12T09:53:29.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.445+0000 7f27f761e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f015e510 con 0x7f27f0157a60 2023-12-12T09:53:29.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.445+0000 7f27f761e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f0071cd0 con 0x7f27f006c410 2023-12-12T09:53:29.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27f5bbb640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f27f015e0e0 0x7f27f00cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39342/0 (socket says 172.21.15.72:39342) 2023-12-12T09:53:29.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27f5bbb640 1 -- 172.21.15.72:0/780235924 learned_addr learned my addr 172.21.15.72:0/780235924 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:29.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27f4bb9640 1 --2- 172.21.15.72:0/780235924 >> v2:172.21.15.113:3300/0 conn(0x7f27f006c410 0x7f27f015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858632281 0 0) 0x7f27f006ca40 con 0x7f27f015e0e0 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27dc003520 con 0x7f27f015e0e0 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584709221 0 0) 0x7f27dc003520 con 0x7f27f015e0e0 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 >> v2:172.21.15.113:3300/0 conn(0x7f27f006c410 msgr2=0x7f27f015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27e7fff640 1 --2- 172.21.15.72:0/780235924 >> v2:172.21.15.113:3300/0 conn(0x7f27f006c410 0x7f27f015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.446+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 >> v1:172.21.15.113:6789/0 conn(0x7f27f0157a60 legacy=0x7f27f006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f0070f10 con 0x7f27f015e0e0 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f27e80019f0 con 0x7f27f015e0e0 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 >> v1:172.21.15.72:6789/0 conn(0x7f27f015e0e0 legacy=0x7f27f00cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27dc0039c0 0x7f27dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc0051d0 0x7f27dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0071cd0 con 0x7f27dc0047c0 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc00c120 con 0x7f27dc0051d0 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc00c2e0 con 0x7f27dc0039c0 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f4bb9640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f4bb9640 1 -- 172.21.15.72:0/780235924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27dc0039c0 msgr2=0x7f27dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f53ba640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27dc0039c0 0x7f27dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f5bbb640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc0051d0 0x7f27dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f4bb9640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27dc0039c0 0x7f27dc003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f4bb9640 1 -- 172.21.15.72:0/780235924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc0051d0 msgr2=0x7f27dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f4bb9640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc0051d0 0x7f27dc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f4bb9640 1 -- 172.21.15.72:0/780235924 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc003520 con 0x7f27dc0047c0 2023-12-12T09:53:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.447+0000 7f27f5bbb640 1 -- 172.21.15.72:0/780235924 reap_dead start 2023-12-12T09:53:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.448+0000 7f27f4bb9640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27dc004c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009a90 tx=0x7f27dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426ab70c4e855831 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.449+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e002c680 con 0x7f27dc0047c0 2023-12-12T09:53:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.449+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e002c7e0 con 0x7f27dc0047c0 2023-12-12T09:53:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.449+0000 7f27e7fff640 1 -- 172.21.15.72:0/780235924 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e002cb70 con 0x7f27dc0047c0 2023-12-12T09:53:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.449+0000 7f27f761e640 1 -- 172.21.15.72:0/780235924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 msgr2=0x7f27dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.449+0000 7f27f761e640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27dc004c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009a90 tx=0x7f27dc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.449+0000 7f27f761e640 1 -- 172.21.15.72:0/780235924 shutdown_connections 2023-12-12T09:53:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.449+0000 7f27f761e640 1 --2- 172.21.15.72:0/780235924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27dc004c40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.449+0000 7f27f761e640 1 -- 172.21.15.72:0/780235924 >> 172.21.15.72:0/780235924 conn(0x7f27f00b0a90 msgr2=0x7f27f00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.450+0000 7f27f761e640 1 -- 172.21.15.72:0/780235924 shutdown_connections 2023-12-12T09:53:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.450+0000 7f27f761e640 1 -- 172.21.15.72:0/780235924 wait complete. 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.451+0000 7f27f761e640 1 Processor -- start 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.451+0000 7f27f761e640 1 -- start start 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.452+0000 7f27f761e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27f0158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.452+0000 7f27f761e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f01480f0 0x7f27f01f0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.452+0000 7f27f761e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27f0149cf0 0x7f27f00afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.452+0000 7f27f761e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0159110 con 0x7f27dc0047c0 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.452+0000 7f27f761e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0158680 con 0x7f27f0149cf0 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.452+0000 7f27f761e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00b4990 con 0x7f27f01480f0 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27f0158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27f0158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44922/0 (socket says 172.21.15.72:44922) 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 -- 172.21.15.72:0/2595293108 learned_addr learned my addr 172.21.15.72:0/2595293108 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 -- 172.21.15.72:0/2595293108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f01480f0 msgr2=0x7f27f01f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f01480f0 0x7f27f01f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 -- 172.21.15.72:0/2595293108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27f0149cf0 msgr2=0x7f27f00afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27f0149cf0 0x7f27f00afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 -- 172.21.15.72:0/2595293108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f01f08b0 con 0x7f27dc0047c0 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.453+0000 7f27f53ba640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27f0158140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00d570 tx=0x7f27ec00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.454+0000 7f27e5ffb640 1 -- 172.21.15.72:0/2595293108 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec005480 con 0x7f27dc0047c0 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.454+0000 7f27e5ffb640 1 -- 172.21.15.72:0/2595293108 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ec003610 con 0x7f27dc0047c0 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.454+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f0158870 con 0x7f27dc0047c0 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.454+0000 7f27e5ffb640 1 -- 172.21.15.72:0/2595293108 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec003980 con 0x7f27dc0047c0 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.454+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f0158e30 con 0x7f27dc0047c0 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.455+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f00745d0 con 0x7f27dc0047c0 2023-12-12T09:53:29.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.456+0000 7f27e5ffb640 1 -- 172.21.15.72:0/2595293108 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27ec003ae0 con 0x7f27dc0047c0 2023-12-12T09:53:29.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.457+0000 7f27e5ffb640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f27d406ae40 0x7f27d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.457+0000 7f27e5ffb640 1 -- 172.21.15.72:0/2595293108 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f27ec089930 con 0x7f27dc0047c0 2023-12-12T09:53:29.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.457+0000 7f27f4bb9640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f27d406ae40 0x7f27d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.458+0000 7f27f4bb9640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f27d406ae40 0x7f27d406d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f27e0036000 tx=0x7f27e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:29.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.461+0000 7f27e5ffb640 1 -- 172.21.15.72:0/2595293108 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27ec054620 con 0x7f27dc0047c0 2023-12-12T09:53:29.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.696+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27f00b19d0 con 0x7f27dc0047c0 2023-12-12T09:53:29.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.697+0000 7f27e5ffb640 1 -- 172.21.15.72:0/2595293108 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f27ec05e020 con 0x7f27dc0047c0 2023-12-12T09:53:29.700 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:29.700 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:35.924557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:53:29.700 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-12-12T09:53:29.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f27d406ae40 msgr2=0x7f27d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f27d406ae40 0x7f27d406d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f27e0036000 tx=0x7f27e002e000 comp rx=0 tx=0).stop 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 msgr2=0x7f27f0158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27f0158140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00d570 tx=0x7f27ec00da30 comp rx=0 tx=0).stop 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 shutdown_connections 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f27d406ae40 0x7f27d406d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27f0149cf0 0x7f27f00afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f01480f0 0x7f27f01f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 --2- 172.21.15.72:0/2595293108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27dc0047c0 0x7f27f0158140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.700+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 >> 172.21.15.72:0/2595293108 conn(0x7f27f00b0a90 msgr2=0x7f27f0156c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.701+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 shutdown_connections 2023-12-12T09:53:29.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.701+0000 7f27f761e640 1 -- 172.21.15.72:0/2595293108 wait complete. 2023-12-12T09:53:29.721 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:29.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.928+0000 7fc72f59e640 1 Processor -- start 2023-12-12T09:53:29.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.928+0000 7fc72f59e640 1 -- start start 2023-12-12T09:53:29.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.929+0000 7fc72f59e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7300b1910 0x7fc7300b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.930+0000 7fc72f59e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73006bf80 con 0x7fc730160f10 2023-12-12T09:53:29.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.930+0000 7fc72f59e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73006c540 con 0x7fc73015d630 2023-12-12T09:53:29.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.930+0000 7fc72f59e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc730161a00 con 0x7fc7300b1910 2023-12-12T09:53:29.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.930+0000 7fc72e59c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc730160f10 0x7fc7301612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39354/0 (socket says 172.21.15.72:39354) 2023-12-12T09:53:29.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.930+0000 7fc72e59c640 1 -- 172.21.15.72:0/3346202020 learned_addr learned my addr 172.21.15.72:0/3346202020 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:29.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.930+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510103515 0 0) 0x7fc73006bf80 con 0x7fc730160f10 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.930+0000 7fc72ed9d640 1 --2- 172.21.15.72:0/3346202020 >> v2:172.21.15.113:3300/0 conn(0x7fc7300b1910 0x7fc7300b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.930+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc71c003520 con 0x7fc730160f10 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2701489026 0 0) 0x7fc71c003520 con 0x7fc730160f10 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 >> v2:172.21.15.113:3300/0 conn(0x7fc7300b1910 msgr2=0x7fc7300b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 --2- 172.21.15.72:0/3346202020 >> v2:172.21.15.113:3300/0 conn(0x7fc7300b1910 0x7fc7300b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 >> v1:172.21.15.113:6789/0 conn(0x7fc73015d630 legacy=0x7fc7300cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730161c90 con 0x7fc730160f10 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fc718001670 con 0x7fc730160f10 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72ed9d640 1 --2- 172.21.15.72:0/3346202020 >> v2:172.21.15.113:3300/0 conn(0x7fc7300b1910 0x7fc7300b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 >> v1:172.21.15.72:6789/0 conn(0x7fc730160f10 legacy=0x7fc7301612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc71c003970 0x7fc71c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc71c0047c0 0x7fc71c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc71c0052a0 0x7fc71c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc730161a00 con 0x7fc71c0052a0 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c00c1f0 con 0x7fc71c0047c0 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c00c3b0 con 0x7fc71c003970 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.931+0000 7fc72ed9d640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc71c0052a0 0x7fc71c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.932+0000 7fc72dd9b640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc71c0047c0 0x7fc71c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.932+0000 7fc72ed9d640 1 -- 172.21.15.72:0/3346202020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc71c003970 msgr2=0x7fc71c003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.932+0000 7fc72ed9d640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc71c003970 0x7fc71c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.932+0000 7fc72ed9d640 1 -- 172.21.15.72:0/3346202020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc71c0047c0 msgr2=0x7fc71c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.932+0000 7fc72ed9d640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc71c0047c0 0x7fc71c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.932+0000 7fc72ed9d640 1 -- 172.21.15.72:0/3346202020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c003520 con 0x7fc71c0052a0 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.932+0000 7fc72ed9d640 1 -- 172.21.15.72:0/3346202020 reap_dead start 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.933+0000 7fc72ed9d640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc71c0052a0 0x7fc71c00b980 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc7280073b0 tx=0x7fc71c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe55da86f3a8170b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.933+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72802cd90 con 0x7fc71c0052a0 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.933+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728034440 con 0x7fc71c0052a0 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.933+0000 7fc72d59a640 1 -- 172.21.15.72:0/3346202020 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7280347f0 con 0x7fc71c0052a0 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.933+0000 7fc72f59e640 1 -- 172.21.15.72:0/3346202020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc71c0052a0 msgr2=0x7fc71c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.933+0000 7fc72f59e640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc71c0052a0 0x7fc71c00b980 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc7280073b0 tx=0x7fc71c0057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.934+0000 7fc72f59e640 1 -- 172.21.15.72:0/3346202020 shutdown_connections 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.934+0000 7fc72f59e640 1 --2- 172.21.15.72:0/3346202020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc71c0052a0 0x7fc71c00b980 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.934+0000 7fc72f59e640 1 -- 172.21.15.72:0/3346202020 >> 172.21.15.72:0/3346202020 conn(0x7fc7300b2800 msgr2=0x7fc7300b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:29.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.934+0000 7fc72f59e640 1 -- 172.21.15.72:0/3346202020 shutdown_connections 2023-12-12T09:53:29.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.934+0000 7fc72f59e640 1 -- 172.21.15.72:0/3346202020 wait complete. 2023-12-12T09:53:29.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.935+0000 7fc72f59e640 1 Processor -- start 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.935+0000 7fc72f59e640 1 -- start start 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.936+0000 7fc72f59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc71c0052a0 0x7fc7300cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.936+0000 7fc72f59e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc730146bf0 0x7fc730147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.936+0000 7fc72f59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc730143e80 0x7fc730144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.936+0000 7fc72f59e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc730144b60 con 0x7fc730146bf0 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.936+0000 7fc72f59e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc730144840 con 0x7fc730143e80 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.936+0000 7fc72f59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7301476b0 con 0x7fc71c0052a0 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc730146bf0 0x7fc730147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc730146bf0 0x7fc730147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44940/0 (socket says 172.21.15.72:44940) 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 -- 172.21.15.72:0/4022873621 learned_addr learned my addr 172.21.15.72:0/4022873621 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 -- 172.21.15.72:0/4022873621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc71c0052a0 msgr2=0x7fc7300cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72e59c640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc71c0052a0 0x7fc7300cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc71c0052a0 0x7fc7300cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72ed9d640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc730143e80 0x7fc730144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 -- 172.21.15.72:0/4022873621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc730143e80 msgr2=0x7fc730144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc730143e80 0x7fc730144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 -- 172.21.15.72:0/4022873621 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730147990 con 0x7fc730146bf0 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.937+0000 7fc72dd9b640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc730146bf0 0x7fc730147050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc72000d4e0 tx=0x7fc72000d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.938+0000 7fc726ffd640 1 -- 172.21.15.72:0/4022873621 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720012800 con 0x7fc730146bf0 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.938+0000 7fc726ffd640 1 -- 172.21.15.72:0/4022873621 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc720012960 con 0x7fc730146bf0 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.938+0000 7fc726ffd640 1 -- 172.21.15.72:0/4022873621 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720012d50 con 0x7fc730146bf0 2023-12-12T09:53:29.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.938+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc730144d70 con 0x7fc730146bf0 2023-12-12T09:53:29.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.938+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7300c74d0 con 0x7fc730146bf0 2023-12-12T09:53:29.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.939+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73006cf80 con 0x7fc730146bf0 2023-12-12T09:53:29.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.940+0000 7fc726ffd640 1 -- 172.21.15.72:0/4022873621 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc720019360 con 0x7fc730146bf0 2023-12-12T09:53:29.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.941+0000 7fc726ffd640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc71006ae40 0x7fc71006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:29.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.941+0000 7fc726ffd640 1 -- 172.21.15.72:0/4022873621 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc720089b30 con 0x7fc730146bf0 2023-12-12T09:53:29.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.941+0000 7fc72e59c640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc71006ae40 0x7fc71006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:29.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.942+0000 7fc72e59c640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc71006ae40 0x7fc71006d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc718001820 tx=0x7fc718009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:29.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:29.945+0000 7fc726ffd640 1 -- 172.21.15.72:0/4022873621 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc720054780 con 0x7fc730146bf0 2023-12-12T09:53:30.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.139+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7300b1340 con 0x7fc730146bf0 2023-12-12T09:53:30.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.141+0000 7fc726ffd640 1 -- 172.21.15.72:0/4022873621 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7fc720020090 con 0x7fc730146bf0 2023-12-12T09:53:30.143 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:30.144 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:30.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc71006ae40 msgr2=0x7fc71006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc71006ae40 0x7fc71006d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc718001820 tx=0x7fc718009040 comp rx=0 tx=0).stop 2023-12-12T09:53:30.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc730146bf0 msgr2=0x7fc730147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc730146bf0 0x7fc730147050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc72000d4e0 tx=0x7fc72000d9a0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 shutdown_connections 2023-12-12T09:53:30.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc71006ae40 0x7fc71006d300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc730143e80 0x7fc730144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc730146bf0 0x7fc730147050 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 --2- 172.21.15.72:0/4022873621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc71c0052a0 0x7fc7300cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 >> 172.21.15.72:0/4022873621 conn(0x7fc7300b2800 msgr2=0x7fc73006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:30.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 shutdown_connections 2023-12-12T09:53:30.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.144+0000 7fc72f59e640 1 -- 172.21.15.72:0/4022873621 wait complete. 2023-12-12T09:53:30.165 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:52:05.675969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:52:06.464147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:52:10.083525+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:53:30.166 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:30.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.378+0000 7f292b82b640 1 Processor -- start 2023-12-12T09:53:30.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.379+0000 7f292b82b640 1 -- start start 2023-12-12T09:53:30.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.380+0000 7f292b82b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f292415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.380+0000 7f292b82b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f292406a460 0x7f292406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.380+0000 7f292b82b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2924071cd0 con 0x7f292415af50 2023-12-12T09:53:30.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.380+0000 7f292b82b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292406ae30 con 0x7f292406a460 2023-12-12T09:53:30.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.380+0000 7f292b82b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29240717e0 con 0x7f29240ce7e0 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.380+0000 7f2929dc8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f292406a460 0x7f292406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f2928dc6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f292415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f2928dc6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f292415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:48532/0 (socket says 172.21.15.72:48532) 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f2929dc8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f292406a460 0x7f292406a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:60486/0 (socket says 172.21.15.72:60486) 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f2928dc6640 1 -- 172.21.15.72:0/3480613906 learned_addr learned my addr 172.21.15.72:0/3480613906 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f29295c7640 1 --1- 172.21.15.72:0/3480613906 >> v1:172.21.15.113:6789/0 conn(0x7f292415af50 0x7f292406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:55048/0 (socket says 172.21.15.72:55048) 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f2929dc8640 1 -- 172.21.15.72:0/3480613906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 msgr2=0x7f292415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f2929dc8640 1 --2- 172.21.15.72:0/3480613906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f292415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f2929dc8640 1 -- 172.21.15.72:0/3480613906 >> v1:172.21.15.113:6789/0 conn(0x7f292415af50 legacy=0x7f292406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.381+0000 7f2929dc8640 1 -- 172.21.15.72:0/3480613906 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240719f0 con 0x7f292406a460 2023-12-12T09:53:30.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.383+0000 7f2929dc8640 1 --2- 172.21.15.72:0/3480613906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292406a460 0x7f292406a8f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f291c009d90 tx=0x7f291c02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81cc9129339f49a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:30.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.383+0000 7f291bfff640 1 -- 172.21.15.72:0/3480613906 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c02cb30 con 0x7f292406a460 2023-12-12T09:53:30.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.383+0000 7f291bfff640 1 -- 172.21.15.72:0/3480613906 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c02cc90 con 0x7f292406a460 2023-12-12T09:53:30.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.383+0000 7f291bfff640 1 -- 172.21.15.72:0/3480613906 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c034440 con 0x7f292406a460 2023-12-12T09:53:30.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.384+0000 7f292b82b640 1 -- 172.21.15.72:0/3480613906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292406a460 msgr2=0x7f292406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.384+0000 7f292b82b640 1 --2- 172.21.15.72:0/3480613906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292406a460 0x7f292406a8f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f291c009d90 tx=0x7f291c02bd80 comp rx=0 tx=0).stop 2023-12-12T09:53:30.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.384+0000 7f292b82b640 1 -- 172.21.15.72:0/3480613906 shutdown_connections 2023-12-12T09:53:30.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.384+0000 7f292b82b640 1 --2- 172.21.15.72:0/3480613906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292406a460 0x7f292406a8f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.384+0000 7f292b82b640 1 --2- 172.21.15.72:0/3480613906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f292415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.384+0000 7f292b82b640 1 -- 172.21.15.72:0/3480613906 >> 172.21.15.72:0/3480613906 conn(0x7f29240cee70 msgr2=0x7f29240cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:30.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.384+0000 7f292b82b640 1 -- 172.21.15.72:0/3480613906 shutdown_connections 2023-12-12T09:53:30.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.384+0000 7f292b82b640 1 -- 172.21.15.72:0/3480613906 wait complete. 2023-12-12T09:53:30.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.386+0000 7f292b82b640 1 Processor -- start 2023-12-12T09:53:30.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.386+0000 7f292b82b640 1 -- start start 2023-12-12T09:53:30.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.387+0000 7f292b82b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f292406a460 0x7f29241ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.387+0000 7f292b82b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f29240c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.387+0000 7f292b82b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292415af50 0x7f29240c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.387+0000 7f292b82b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240b1d90 con 0x7f292406a460 2023-12-12T09:53:30.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.387+0000 7f292b82b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240c8ba0 con 0x7f292415af50 2023-12-12T09:53:30.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.387+0000 7f292b82b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29240b1a60 con 0x7f29240ce7e0 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.387+0000 7f29295c7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f292406a460 0x7f29241ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f29295c7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f292406a460 0x7f29241ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44944/0 (socket says 172.21.15.72:44944) 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f29295c7640 1 -- 172.21.15.72:0/3463519886 learned_addr learned my addr 172.21.15.72:0/3463519886 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f2929dc8640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292415af50 0x7f29240c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f2928dc6640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f29240c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f29295c7640 1 -- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 msgr2=0x7f29240c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f29295c7640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f29240c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f29295c7640 1 -- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292415af50 msgr2=0x7f29240c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f29295c7640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292415af50 0x7f29240c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f29295c7640 1 -- 172.21.15.72:0/3463519886 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c009a70 con 0x7f292406a460 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f2928dc6640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f29240c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:30.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.388+0000 7f29295c7640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f292406a460 0x7f29241ee0c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f292000c4d0 tx=0x7f292000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:30.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.389+0000 7f2919ffb640 1 -- 172.21.15.72:0/3463519886 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920007480 con 0x7f292406a460 2023-12-12T09:53:30.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.389+0000 7f2919ffb640 1 -- 172.21.15.72:0/3463519886 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29200075e0 con 0x7f292406a460 2023-12-12T09:53:30.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.389+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29240c85e0 con 0x7f292406a460 2023-12-12T09:53:30.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.389+0000 7f2919ffb640 1 -- 172.21.15.72:0/3463519886 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29200049e0 con 0x7f292406a460 2023-12-12T09:53:30.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.389+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29241ee600 con 0x7f292406a460 2023-12-12T09:53:30.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.390+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2924074450 con 0x7f292406a460 2023-12-12T09:53:30.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.391+0000 7f2919ffb640 1 -- 172.21.15.72:0/3463519886 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2920004b40 con 0x7f292406a460 2023-12-12T09:53:30.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.392+0000 7f2919ffb640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f290806ae40 0x7f290806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.392+0000 7f2919ffb640 1 -- 172.21.15.72:0/3463519886 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f29200889c0 con 0x7f292406a460 2023-12-12T09:53:30.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.392+0000 7f2928dc6640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f290806ae40 0x7f290806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.393+0000 7f2928dc6640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f290806ae40 0x7f290806d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29240c6f80 tx=0x7f2914008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:30.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.396+0000 7f2919ffb640 1 -- 172.21.15.72:0/3463519886 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29200537f0 con 0x7f292406a460 2023-12-12T09:53:30.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.584+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f292415a750 con 0x7f292406a460 2023-12-12T09:53:30.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.586+0000 7f2919ffb640 1 -- 172.21.15.72:0/3463519886 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29200180a0 con 0x7f292406a460 2023-12-12T09:53:30.587 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:30.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f290806ae40 msgr2=0x7f290806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f290806ae40 0x7f290806d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29240c6f80 tx=0x7f2914008040 comp rx=0 tx=0).stop 2023-12-12T09:53:30.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f292406a460 msgr2=0x7f29241ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f292406a460 0x7f29241ee0c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f292000c4d0 tx=0x7f292000c990 comp rx=0 tx=0).stop 2023-12-12T09:53:30.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 shutdown_connections 2023-12-12T09:53:30.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f290806ae40 0x7f290806d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f292415af50 0x7f29240c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29240ce7e0 0x7f29240c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 --2- 172.21.15.72:0/3463519886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f292406a460 0x7f29241ee0c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 >> 172.21.15.72:0/3463519886 conn(0x7f29240cee70 msgr2=0x7f292406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 shutdown_connections 2023-12-12T09:53:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.588+0000 7f292b82b640 1 -- 172.21.15.72:0/3463519886 wait complete. 2023-12-12T09:53:30.609 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:53:30.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.819+0000 7fb922cdf640 1 Processor -- start 2023-12-12T09:53:30.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.819+0000 7fb922cdf640 1 -- start start 2023-12-12T09:53:30.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.821+0000 7fb922cdf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c15ab60 0x7fb91c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.821+0000 7fb922cdf640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb91c0b3ca0 0x7fb91c1610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.821+0000 7fb922cdf640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91c1615e0 con 0x7fb91c160c60 2023-12-12T09:53:30.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.821+0000 7fb922cdf640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91c0cb3a0 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.821+0000 7fb922cdf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0cb680 con 0x7fb91c15ab60 2023-12-12T09:53:30.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.821+0000 7fb9224de640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb91c0b3ca0 0x7fb91c1610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.821+0000 7fb9224de640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb91c0b3ca0 0x7fb91c1610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44958/0 (socket says 172.21.15.72:44958) 2023-12-12T09:53:30.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.821+0000 7fb9224de640 1 -- 172.21.15.72:0/535628512 learned_addr learned my addr 172.21.15.72:0/535628512 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:30.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.822+0000 7fb921cdd640 1 --2- 172.21.15.72:0/535628512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c15ab60 0x7fb91c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.822+0000 7fb9224de640 1 -- 172.21.15.72:0/535628512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c15ab60 msgr2=0x7fb91c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:30.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.822+0000 7fb9224de640 1 --2- 172.21.15.72:0/535628512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c15ab60 0x7fb91c0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.822+0000 7fb9224de640 1 -- 172.21.15.72:0/535628512 >> v1:172.21.15.113:6789/0 conn(0x7fb91c160c60 legacy=0x7fb91c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:30.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.822+0000 7fb9224de640 1 -- 172.21.15.72:0/535628512 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c0cb960 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.822+0000 7fb921cdd640 1 --2- 172.21.15.72:0/535628512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c15ab60 0x7fb91c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:30.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.823+0000 7fb9224de640 1 --2- 172.21.15.72:0/535628512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c1610a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb914009e50 tx=0x7fb91402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a14dd0417234856 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:30.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.824+0000 7fb920cdb640 1 -- 172.21.15.72:0/535628512 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91402ca80 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.824+0000 7fb920cdb640 1 -- 172.21.15.72:0/535628512 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91402cbe0 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.824+0000 7fb920cdb640 1 -- 172.21.15.72:0/535628512 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9140343f0 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.824+0000 7fb922cdf640 1 -- 172.21.15.72:0/535628512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 msgr2=0x7fb91c1610a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.824+0000 7fb922cdf640 1 --2- 172.21.15.72:0/535628512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c1610a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb914009e50 tx=0x7fb91402bee0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.825+0000 7fb922cdf640 1 -- 172.21.15.72:0/535628512 shutdown_connections 2023-12-12T09:53:30.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.825+0000 7fb922cdf640 1 --2- 172.21.15.72:0/535628512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c1610a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.825+0000 7fb922cdf640 1 --2- 172.21.15.72:0/535628512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c15ab60 0x7fb91c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.825+0000 7fb922cdf640 1 -- 172.21.15.72:0/535628512 >> 172.21.15.72:0/535628512 conn(0x7fb91c0b0940 msgr2=0x7fb91c0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:30.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.825+0000 7fb922cdf640 1 -- 172.21.15.72:0/535628512 shutdown_connections 2023-12-12T09:53:30.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.825+0000 7fb922cdf640 1 -- 172.21.15.72:0/535628512 wait complete. 2023-12-12T09:53:30.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.827+0000 7fb922cdf640 1 Processor -- start 2023-12-12T09:53:30.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.827+0000 7fb922cdf640 1 -- start start 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.828+0000 7fb922cdf640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb922cdf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb91c15ab60 0x7fb91c0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb922cdf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c160c60 0x7fb91c0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb922cdf640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0cac20 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb922cdf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c148b60 con 0x7fb91c15ab60 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb922cdf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0c9fa0 con 0x7fb91c160c60 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c148620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44966/0 (socket says 172.21.15.72:44966) 2023-12-12T09:53:30.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 -- 172.21.15.72:0/3332672271 learned_addr learned my addr 172.21.15.72:0/3332672271 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:30.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 -- 172.21.15.72:0/3332672271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c160c60 msgr2=0x7fb91c0aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c160c60 0x7fb91c0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 -- 172.21.15.72:0/3332672271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb91c15ab60 msgr2=0x7fb91c0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:30.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb91c15ab60 0x7fb91c0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:30.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 -- 172.21.15.72:0/3332672271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914009b30 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.829+0000 7fb921cdd640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c148620 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb91800c7f0 tx=0x7fb91800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:30.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.830+0000 7fb9127fc640 1 -- 172.21.15.72:0/3332672271 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180079b0 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.830+0000 7fb9127fc640 1 -- 172.21.15.72:0/3332672271 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb918007b10 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.830+0000 7fb9127fc640 1 -- 172.21.15.72:0/3332672271 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918007e80 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.830+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c0ca280 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.830+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c0ca840 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.831+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c067910 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.832+0000 7fb9127fc640 1 -- 172.21.15.72:0/3332672271 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9180113f0 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.833+0000 7fb9127fc640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb90006ae70 0x7fb90006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:30.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.833+0000 7fb9127fc640 1 -- 172.21.15.72:0/3332672271 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb918089b60 con 0x7fb91c0b3ca0 2023-12-12T09:53:30.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.833+0000 7fb9214dc640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb90006ae70 0x7fb90006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:30.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.834+0000 7fb9214dc640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb90006ae70 0x7fb90006d330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006fd0 tx=0x7fb90c006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:30.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:30.837+0000 7fb9127fc640 1 -- 172.21.15.72:0/3332672271 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb918054850 con 0x7fb91c0b3ca0 2023-12-12T09:53:31.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.028+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb91c074450 con 0x7fb91c0b3ca0 2023-12-12T09:53:31.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.029+0000 7fb9127fc640 1 -- 172.21.15.72:0/3332672271 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb918057e70 con 0x7fb91c0b3ca0 2023-12-12T09:53:31.031 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:31.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb90006ae70 msgr2=0x7fb90006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb90006ae70 0x7fb90006d330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006fd0 tx=0x7fb90c006bc0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 msgr2=0x7fb91c148620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c148620 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb91800c7f0 tx=0x7fb91800cbf0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 shutdown_connections 2023-12-12T09:53:31.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb90006ae70 0x7fb90006d330 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb91c160c60 0x7fb91c0aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb91c15ab60 0x7fb91c0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 --2- 172.21.15.72:0/3332672271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb91c0b3ca0 0x7fb91c148620 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.032+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 >> 172.21.15.72:0/3332672271 conn(0x7fb91c0b0940 msgr2=0x7fb91c0b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:31.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.033+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 shutdown_connections 2023-12-12T09:53:31.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.033+0000 7fb922cdf640 1 -- 172.21.15.72:0/3332672271 wait complete. 2023-12-12T09:53:31.053 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:53:31.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.265+0000 7fcc300d3640 1 Processor -- start 2023-12-12T09:53:31.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.265+0000 7fcc300d3640 1 -- start start 2023-12-12T09:53:31.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.266+0000 7fcc300d3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcc28071fb0 0x7fcc280ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.266+0000 7fcc300d3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280ca4f0 0x7fcc280c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.266+0000 7fcc300d3640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280caf80 con 0x7fcc280c21c0 2023-12-12T09:53:31.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.266+0000 7fcc300d3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc280ca930 con 0x7fcc28071fb0 2023-12-12T09:53:31.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.266+0000 7fcc300d3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280c2650 con 0x7fcc280ca4f0 2023-12-12T09:53:31.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2e670640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fcc280c21c0 0x7fcc280cc860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39366/0 (socket says 172.21.15.72:39366) 2023-12-12T09:53:31.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2e670640 1 -- 172.21.15.72:0/1483161497 learned_addr learned my addr 172.21.15.72:0/1483161497 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2de6f640 1 --2- 172.21.15.72:0/1483161497 >> v2:172.21.15.113:3300/0 conn(0x7fcc28071fb0 0x7fcc280ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2d66e640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280ca4f0 0x7fcc280c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649433731 0 0) 0x7fcc280caf80 con 0x7fcc280c21c0 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc14003520 con 0x7fcc280c21c0 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1512415657 0 0) 0x7fcc14003520 con 0x7fcc280c21c0 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280ca4f0 msgr2=0x7fcc280c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280ca4f0 0x7fcc280c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 >> v2:172.21.15.113:3300/0 conn(0x7fcc28071fb0 msgr2=0x7fcc280ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 --2- 172.21.15.72:0/1483161497 >> v2:172.21.15.113:3300/0 conn(0x7fcc28071fb0 0x7fcc280ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2806a5e0 con 0x7fcc280c21c0 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.267+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fcc200019f0 con 0x7fcc280c21c0 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 >> v1:172.21.15.72:6789/0 conn(0x7fcc280c21c0 legacy=0x7fcc280cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2ce6d640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280ca4f0 0x7fcc14003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2ce6d640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc14003ef0 0x7fcc140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2ce6d640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14004d00 0x7fcc14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280ca930 con 0x7fcc14003ef0 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280c2650 con 0x7fcc14004d00 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1400b980 con 0x7fcc280ca4f0 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2d66e640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc14003ef0 0x7fcc140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2de6f640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280ca4f0 0x7fcc14003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2d66e640 1 -- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280ca4f0 msgr2=0x7fcc14003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2d66e640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280ca4f0 0x7fcc14003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2e670640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14004d00 0x7fcc14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.268+0000 7fcc2d66e640 1 -- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14004d00 msgr2=0x7fcc14005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.269+0000 7fcc2d66e640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14004d00 0x7fcc14005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.269+0000 7fcc2d66e640 1 -- 172.21.15.72:0/1483161497 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc14003520 con 0x7fcc14003ef0 2023-12-12T09:53:31.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.269+0000 7fcc2e670640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14004d00 0x7fcc14005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:31.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.270+0000 7fcc2d66e640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc14003ef0 0x7fcc140047c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009a90 tx=0x7fcc140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149eb0dcaa604c0a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.270+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1802c6b0 con 0x7fcc14003ef0 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.270+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1802c810 con 0x7fcc14003ef0 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.270+0000 7fcc2ce6d640 1 -- 172.21.15.72:0/1483161497 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1802cbc0 con 0x7fcc14003ef0 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.270+0000 7fcc300d3640 1 -- 172.21.15.72:0/1483161497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc14003ef0 msgr2=0x7fcc140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.270+0000 7fcc300d3640 1 --2- 172.21.15.72:0/1483161497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc14003ef0 0x7fcc140047c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009a90 tx=0x7fcc140057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.270+0000 7fcc2e670640 1 -- 172.21.15.72:0/1483161497 reap_dead start 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.271+0000 7fcc300d3640 1 -- 172.21.15.72:0/1483161497 shutdown_connections 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.271+0000 7fcc300d3640 1 -- 172.21.15.72:0/1483161497 >> 172.21.15.72:0/1483161497 conn(0x7fcc280cee70 msgr2=0x7fcc280cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.271+0000 7fcc300d3640 1 -- 172.21.15.72:0/1483161497 shutdown_connections 2023-12-12T09:53:31.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.271+0000 7fcc300d3640 1 -- 172.21.15.72:0/1483161497 wait complete. 2023-12-12T09:53:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.273+0000 7fcc300d3640 1 Processor -- start 2023-12-12T09:53:31.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.273+0000 7fcc300d3640 1 -- start start 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.274+0000 7fcc300d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14003ef0 0x7fcc2815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.274+0000 7fcc300d3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280c5340 0x7fcc280999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.274+0000 7fcc300d3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc2809b400 0x7fcc2806c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc300d3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2815a820 con 0x7fcc2809b400 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc300d3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2809a360 con 0x7fcc14003ef0 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc300d3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2809a640 con 0x7fcc280c5340 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2e670640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc2809b400 0x7fcc2806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2e670640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc2809b400 0x7fcc2806c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44994/0 (socket says 172.21.15.72:44994) 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2e670640 1 -- 172.21.15.72:0/492800902 learned_addr learned my addr 172.21.15.72:0/492800902 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:31.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2de6f640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14003ef0 0x7fcc2815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2d66e640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280c5340 0x7fcc280999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2e670640 1 -- 172.21.15.72:0/492800902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280c5340 msgr2=0x7fcc280999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2e670640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280c5340 0x7fcc280999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2e670640 1 -- 172.21.15.72:0/492800902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14003ef0 msgr2=0x7fcc2815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2e670640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14003ef0 0x7fcc2815a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.275+0000 7fcc2e670640 1 -- 172.21.15.72:0/492800902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2809a920 con 0x7fcc2809b400 2023-12-12T09:53:31.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.276+0000 7fcc2e670640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc2809b400 0x7fcc2806c200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcc20000f80 tx=0x7fcc2000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:31.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.276+0000 7fcc1e7fc640 1 -- 172.21.15.72:0/492800902 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000deb0 con 0x7fcc2809b400 2023-12-12T09:53:31.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.276+0000 7fcc1e7fc640 1 -- 172.21.15.72:0/492800902 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2000bb80 con 0x7fcc2809b400 2023-12-12T09:53:31.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.276+0000 7fcc1e7fc640 1 -- 172.21.15.72:0/492800902 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20007070 con 0x7fcc2809b400 2023-12-12T09:53:31.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.276+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2809ac00 con 0x7fcc2809b400 2023-12-12T09:53:31.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.276+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc28099f30 con 0x7fcc2809b400 2023-12-12T09:53:31.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.278+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2809b120 con 0x7fcc2809b400 2023-12-12T09:53:31.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.278+0000 7fcc1e7fc640 1 -- 172.21.15.72:0/492800902 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc200071d0 con 0x7fcc2809b400 2023-12-12T09:53:31.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.279+0000 7fcc1e7fc640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc0c06ae40 0x7fcc0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.279+0000 7fcc1e7fc640 1 -- 172.21.15.72:0/492800902 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcc20089e60 con 0x7fcc2809b400 2023-12-12T09:53:31.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.279+0000 7fcc2de6f640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc0c06ae40 0x7fcc0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.280+0000 7fcc2de6f640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc0c06ae40 0x7fcc0c06d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009a90 tx=0x7fcc24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:31.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.285+0000 7fcc1e7fc640 1 -- 172.21.15.72:0/492800902 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc2809b120 con 0x7fcc2809b400 2023-12-12T09:53:31.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.478+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc2809b120 con 0x7fcc2809b400 2023-12-12T09:53:31.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.479+0000 7fcc1e7fc640 1 -- 172.21.15.72:0/492800902 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc2809b120 con 0x7fcc2809b400 2023-12-12T09:53:31.480 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:53:31.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc0c06ae40 msgr2=0x7fcc0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc0c06ae40 0x7fcc0c06d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009a90 tx=0x7fcc24008040 comp rx=0 tx=0).stop 2023-12-12T09:53:31.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc2809b400 msgr2=0x7fcc2806c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc2809b400 0x7fcc2806c200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcc20000f80 tx=0x7fcc2000b4a0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 shutdown_connections 2023-12-12T09:53:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc0c06ae40 0x7fcc0c06d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc2809b400 0x7fcc2806c200 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc280c5340 0x7fcc280999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 --2- 172.21.15.72:0/492800902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc14003ef0 0x7fcc2815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.481+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 >> 172.21.15.72:0/492800902 conn(0x7fcc280cee70 msgr2=0x7fcc28160200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.482+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 shutdown_connections 2023-12-12T09:53:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.482+0000 7fcc300d3640 1 -- 172.21.15.72:0/492800902 wait complete. 2023-12-12T09:53:31.503 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:53:31.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.717+0000 7f7988bea640 1 Processor -- start 2023-12-12T09:53:31.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.717+0000 7f7988bea640 1 -- start start 2023-12-12T09:53:31.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7988bea640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f798406db50 0x7f7984141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7988bea640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 0x7f79840cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7988bea640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798415b780 con 0x7f798406d770 2023-12-12T09:53:31.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7988bea640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7984141c20 con 0x7f798406db50 2023-12-12T09:53:31.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7988bea640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840b2510 con 0x7f7984141790 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7981d9b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f798406db50 0x7f7984141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7981d9b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f798406db50 0x7f7984141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45004/0 (socket says 172.21.15.72:45004) 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7982d9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 0x7f79840cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.719+0000 7f7981d9b640 1 -- 172.21.15.72:0/4138254438 learned_addr learned my addr 172.21.15.72:0/4138254438 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.720+0000 7f7981d9b640 1 -- 172.21.15.72:0/4138254438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 msgr2=0x7f79840cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.720+0000 7f7981d9b640 1 --2- 172.21.15.72:0/4138254438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 0x7f79840cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.720+0000 7f7981d9b640 1 -- 172.21.15.72:0/4138254438 >> v1:172.21.15.113:6789/0 conn(0x7f798406d770 legacy=0x7f79840b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.720+0000 7f7981d9b640 1 -- 172.21.15.72:0/4138254438 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798415b9d0 con 0x7f798406db50 2023-12-12T09:53:31.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.721+0000 7f7981d9b640 1 --2- 172.21.15.72:0/4138254438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f7984141250 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7978009870 tx=0x7f797802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10550f2e3f9889b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:31.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.721+0000 7f798159a640 1 -- 172.21.15.72:0/4138254438 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978030840 con 0x7f798406db50 2023-12-12T09:53:31.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.721+0000 7f798159a640 1 -- 172.21.15.72:0/4138254438 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79780309a0 con 0x7f798406db50 2023-12-12T09:53:31.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.721+0000 7f798159a640 1 -- 172.21.15.72:0/4138254438 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978030d30 con 0x7f798406db50 2023-12-12T09:53:31.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.722+0000 7f7988bea640 1 -- 172.21.15.72:0/4138254438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 msgr2=0x7f7984141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.722+0000 7f7988bea640 1 --2- 172.21.15.72:0/4138254438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f7984141250 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7978009870 tx=0x7f797802b980 comp rx=0 tx=0).stop 2023-12-12T09:53:31.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.722+0000 7f7988bea640 1 -- 172.21.15.72:0/4138254438 shutdown_connections 2023-12-12T09:53:31.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.722+0000 7f7988bea640 1 --2- 172.21.15.72:0/4138254438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 0x7f79840cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.722+0000 7f7988bea640 1 --2- 172.21.15.72:0/4138254438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f7984141250 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.722+0000 7f7988bea640 1 -- 172.21.15.72:0/4138254438 >> 172.21.15.72:0/4138254438 conn(0x7f79840cb850 msgr2=0x7f79840cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:31.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.722+0000 7f7988bea640 1 -- 172.21.15.72:0/4138254438 shutdown_connections 2023-12-12T09:53:31.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.722+0000 7f7988bea640 1 -- 172.21.15.72:0/4138254438 wait complete. 2023-12-12T09:53:31.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.724+0000 7f7988bea640 1 Processor -- start 2023-12-12T09:53:31.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.724+0000 7f7988bea640 1 -- start start 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7988bea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798406d770 0x7f79841e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7988bea640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f79840af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7988bea640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 0x7f79840ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7988bea640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79841e87f0 con 0x7f798406db50 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7988bea640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840af740 con 0x7f798406d770 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7988bea640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79841e7c20 con 0x7f7984141790 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7981d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f79840af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7981d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f79840af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45016/0 (socket says 172.21.15.72:45016) 2023-12-12T09:53:31.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.725+0000 7f7981d9b640 1 -- 172.21.15.72:0/3518346182 learned_addr learned my addr 172.21.15.72:0/3518346182 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:31.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f798259c640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798406d770 0x7f79841e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7982d9d640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 0x7f79840ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7981d9b640 1 -- 172.21.15.72:0/3518346182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 msgr2=0x7f79840ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7981d9b640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 0x7f79840ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7981d9b640 1 -- 172.21.15.72:0/3518346182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798406d770 msgr2=0x7f79841e63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7981d9b640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798406d770 0x7f79841e63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7981d9b640 1 -- 172.21.15.72:0/3518346182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79780094d0 con 0x7f798406db50 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7981d9b640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f79840af200 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f797802b9c0 tx=0x7f7978004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7972ffd640 1 -- 172.21.15.72:0/3518346182 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978039c90 con 0x7f798406db50 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7972ffd640 1 -- 172.21.15.72:0/3518346182 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978039df0 con 0x7f798406db50 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7972ffd640 1 -- 172.21.15.72:0/3518346182 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780446c0 con 0x7f798406db50 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.726+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79840ada80 con 0x7f798406db50 2023-12-12T09:53:31.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.727+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79841e7e50 con 0x7f798406db50 2023-12-12T09:53:31.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.728+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79840b4930 con 0x7f798406db50 2023-12-12T09:53:31.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.728+0000 7f7972ffd640 1 -- 172.21.15.72:0/3518346182 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f797802f390 con 0x7f798406db50 2023-12-12T09:53:31.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.729+0000 7f7972ffd640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f796406ae70 0x7f796406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:31.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.729+0000 7f7972ffd640 1 -- 172.21.15.72:0/3518346182 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f797803e070 con 0x7f798406db50 2023-12-12T09:53:31.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.730+0000 7f798259c640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f796406ae70 0x7f796406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:31.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.730+0000 7f798259c640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f796406ae70 0x7f796406d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7974000f60 tx=0x7f7974009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:31.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.734+0000 7f7972ffd640 1 -- 172.21.15.72:0/3518346182 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79780dee90 con 0x7f798406db50 2023-12-12T09:53:31.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.924+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f79840745d0 con 0x7f798406db50 2023-12-12T09:53:31.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.924+0000 7f7972ffd640 1 -- 172.21.15.72:0/3518346182 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f79780340a0 con 0x7f798406db50 2023-12-12T09:53:31.926 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:53:31.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f796406ae70 msgr2=0x7f796406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f796406ae70 0x7f796406d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7974000f60 tx=0x7f7974009040 comp rx=0 tx=0).stop 2023-12-12T09:53:31.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 msgr2=0x7f79840af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:31.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f79840af200 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f797802b9c0 tx=0x7f7978004730 comp rx=0 tx=0).stop 2023-12-12T09:53:31.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 shutdown_connections 2023-12-12T09:53:31.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f796406ae70 0x7f796406d330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7984141790 0x7f79840ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f798406db50 0x7f79840af200 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 --2- 172.21.15.72:0/3518346182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798406d770 0x7f79841e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:31.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 >> 172.21.15.72:0/3518346182 conn(0x7f79840cb850 msgr2=0x7f798415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:31.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 shutdown_connections 2023-12-12T09:53:31.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:31.927+0000 7f7988bea640 1 -- 172.21.15.72:0/3518346182 wait complete. 2023-12-12T09:53:31.950 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:53:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.166+0000 7fc5f5c67640 1 Processor -- start 2023-12-12T09:53:32.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.167+0000 7fc5f5c67640 1 -- start start 2023-12-12T09:53:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.168+0000 7fc5f5c67640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc5f015e5d0 0x7fc5f006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.168+0000 7fc5f5c67640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc5f00ae200 0x7fc5f0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.168+0000 7fc5f5c67640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f006c330 0x7fc5f0071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.168+0000 7fc5f5c67640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f0071f90 con 0x7fc5f00ae200 2023-12-12T09:53:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.168+0000 7fc5f5c67640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f0072270 con 0x7fc5f015e5d0 2023-12-12T09:53:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.168+0000 7fc5f5c67640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0070070 con 0x7fc5f006c330 2023-12-12T09:53:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5effff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc5f00ae200 0x7fc5f0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5effff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc5f00ae200 0x7fc5f0070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45024/0 (socket says 172.21.15.72:45024) 2023-12-12T09:53:32.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5effff640 1 -- 172.21.15.72:0/1763079752 learned_addr learned my addr 172.21.15.72:0/1763079752 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:32.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5f4c65640 1 --2- 172.21.15.72:0/1763079752 >> v2:172.21.15.113:3300/0 conn(0x7fc5f015e5d0 0x7fc5f006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5effff640 1 -- 172.21.15.72:0/1763079752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f006c330 msgr2=0x7fc5f0071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:32.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5f5466640 1 --2- 172.21.15.72:0/1763079752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f006c330 0x7fc5f0071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5effff640 1 --2- 172.21.15.72:0/1763079752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f006c330 0x7fc5f0071090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5effff640 1 -- 172.21.15.72:0/1763079752 >> v2:172.21.15.113:3300/0 conn(0x7fc5f015e5d0 msgr2=0x7fc5f006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:32.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5effff640 1 --2- 172.21.15.72:0/1763079752 >> v2:172.21.15.113:3300/0 conn(0x7fc5f015e5d0 0x7fc5f006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.169+0000 7fc5effff640 1 -- 172.21.15.72:0/1763079752 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0070350 con 0x7fc5f00ae200 2023-12-12T09:53:32.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.170+0000 7fc5effff640 1 --2- 172.21.15.72:0/1763079752 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0070b50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5f006bb50 tx=0x7fc5dc02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274e6d86feb869a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:32.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5ef7fe640 1 -- 172.21.15.72:0/1763079752 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc031750 con 0x7fc5f00ae200 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5ef7fe640 1 -- 172.21.15.72:0/1763079752 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc0318b0 con 0x7fc5f00ae200 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5ef7fe640 1 -- 172.21.15.72:0/1763079752 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc031c40 con 0x7fc5f00ae200 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1763079752 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 msgr2=0x7fc5f0070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1763079752 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0070b50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5f006bb50 tx=0x7fc5dc02b7f0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1763079752 shutdown_connections 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1763079752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f006c330 0x7fc5f0071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1763079752 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0070b50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1763079752 >> v2:172.21.15.113:3300/0 conn(0x7fc5f015e5d0 0x7fc5f006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1763079752 >> 172.21.15.72:0/1763079752 conn(0x7fc5f00b0860 msgr2=0x7fc5f00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1763079752 shutdown_connections 2023-12-12T09:53:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.171+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1763079752 wait complete. 2023-12-12T09:53:32.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.173+0000 7fc5f5c67640 1 Processor -- start 2023-12-12T09:53:32.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.173+0000 7fc5f5c67640 1 -- start start 2023-12-12T09:53:32.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.174+0000 7fc5f5c67640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f006c330 0x7fc5f01f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.174+0000 7fc5f5c67640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.174+0000 7fc5f5c67640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f015e5d0 0x7fc5f014db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.174+0000 7fc5f5c67640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f014e560 con 0x7fc5f00ae200 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.174+0000 7fc5f5c67640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01f2f40 con 0x7fc5f006c330 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.174+0000 7fc5f5c67640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0099b90 con 0x7fc5f015e5d0 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.174+0000 7fc5effff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.174+0000 7fc5effff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45026/0 (socket says 172.21.15.72:45026) 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5effff640 1 -- 172.21.15.72:0/1618860650 learned_addr learned my addr 172.21.15.72:0/1618860650 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5f5466640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f015e5d0 0x7fc5f014db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5f4c65640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f006c330 0x7fc5f01f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5effff640 1 -- 172.21.15.72:0/1618860650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f015e5d0 msgr2=0x7fc5f014db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5effff640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f015e5d0 0x7fc5f014db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5effff640 1 -- 172.21.15.72:0/1618860650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f006c330 msgr2=0x7fc5f01f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5effff640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f006c330 0x7fc5f01f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5effff640 1 -- 172.21.15.72:0/1618860650 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc0094d0 con 0x7fc5f00ae200 2023-12-12T09:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5effff640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0144470 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc02b830 tx=0x7fc5dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5ed7fa640 1 -- 172.21.15.72:0/1618860650 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc033c80 con 0x7fc5f00ae200 2023-12-12T09:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.175+0000 7fc5ed7fa640 1 -- 172.21.15.72:0/1618860650 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc033e00 con 0x7fc5f00ae200 2023-12-12T09:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.176+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f0099e70 con 0x7fc5f00ae200 2023-12-12T09:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.176+0000 7fc5ed7fa640 1 -- 172.21.15.72:0/1618860650 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc03c640 con 0x7fc5f00ae200 2023-12-12T09:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.176+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f009a430 con 0x7fc5f00ae200 2023-12-12T09:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.177+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0074450 con 0x7fc5f00ae200 2023-12-12T09:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.178+0000 7fc5ed7fa640 1 -- 172.21.15.72:0/1618860650 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc5dc03a040 con 0x7fc5f00ae200 2023-12-12T09:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.178+0000 7fc5ed7fa640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc5d406ae40 0x7fc5d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.179+0000 7fc5ed7fa640 1 -- 172.21.15.72:0/1618860650 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc5dc0ab6b0 con 0x7fc5f00ae200 2023-12-12T09:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.179+0000 7fc5f4c65640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc5d406ae40 0x7fc5d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.179+0000 7fc5f4c65640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc5d406ae40 0x7fc5d406d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4006fd0 tx=0x7fc5e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:32.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.183+0000 7fc5ed7fa640 1 -- 172.21.15.72:0/1618860650 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5dc076300 con 0x7fc5f00ae200 2023-12-12T09:53:32.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.392+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5f00adfe0 con 0x7fc5f00ae200 2023-12-12T09:53:32.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.393+0000 7fc5ed7fa640 1 -- 172.21.15.72:0/1618860650 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fc5dc079920 con 0x7fc5f00ae200 2023-12-12T09:53:32.395 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:32.396 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:35.924557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:53:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc5d406ae40 msgr2=0x7fc5d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc5d406ae40 0x7fc5d406d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4006fd0 tx=0x7fc5e4008040 comp rx=0 tx=0).stop 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 msgr2=0x7fc5f0144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0144470 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc02b830 tx=0x7fc5dc0046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 shutdown_connections 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc5d406ae40 0x7fc5d406d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5f015e5d0 0x7fc5f014db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5f00ae200 0x7fc5f0144470 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 --2- 172.21.15.72:0/1618860650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f006c330 0x7fc5f01f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.396+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 >> 172.21.15.72:0/1618860650 conn(0x7fc5f00b0860 msgr2=0x7fc5f006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:32.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.397+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 shutdown_connections 2023-12-12T09:53:32.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.397+0000 7fc5f5c67640 1 -- 172.21.15.72:0/1618860650 wait complete. 2023-12-12T09:53:32.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:32.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.600+0000 7f9466d83640 1 Processor -- start 2023-12-12T09:53:32.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.600+0000 7f9466d83640 1 -- start start 2023-12-12T09:53:32.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.602+0000 7f9466d83640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9460143e00 0x7f94600b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.602+0000 7f9466d83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946006c410 0x7f946015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.602+0000 7f9466d83640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600b3bb0 con 0x7f9460072180 2023-12-12T09:53:32.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.602+0000 7f9466d83640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94601441e0 con 0x7f9460143e00 2023-12-12T09:53:32.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.602+0000 7f9466d83640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9460072680 con 0x7f946006c410 2023-12-12T09:53:32.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.603+0000 7f9465320640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946006c410 0x7f946015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.603+0000 7f9464b1f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9460143e00 0x7f94600b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.603+0000 7f9465320640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946006c410 0x7f946015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:48606/0 (socket says 172.21.15.72:48606) 2023-12-12T09:53:32.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.603+0000 7f9465320640 1 -- 172.21.15.72:0/1420662014 learned_addr learned my addr 172.21.15.72:0/1420662014 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:32.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.604+0000 7f9464b1f640 1 -- 172.21.15.72:0/1420662014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946006c410 msgr2=0x7f946015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.604+0000 7f9464b1f640 1 --2- 172.21.15.72:0/1420662014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946006c410 0x7f946015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.604+0000 7f9464b1f640 1 -- 172.21.15.72:0/1420662014 >> v1:172.21.15.113:6789/0 conn(0x7f9460072180 legacy=0x7f94600b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.604+0000 7f9464b1f640 1 -- 172.21.15.72:0/1420662014 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600b1e30 con 0x7f9460143e00 2023-12-12T09:53:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.604+0000 7f9465320640 1 --2- 172.21.15.72:0/1420662014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946006c410 0x7f946015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:53:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.605+0000 7f9464b1f640 1 --2- 172.21.15.72:0/1420662014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 0x7f94600b4040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94540097a0 tx=0x7f945402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79231b03fb4333 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:32.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.612+0000 7f945f7fe640 1 -- 172.21.15.72:0/1420662014 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945402d040 con 0x7f9460143e00 2023-12-12T09:53:32.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.612+0000 7f945f7fe640 1 -- 172.21.15.72:0/1420662014 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454031810 con 0x7f9460143e00 2023-12-12T09:53:32.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.612+0000 7f945f7fe640 1 -- 172.21.15.72:0/1420662014 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454031ba0 con 0x7f9460143e00 2023-12-12T09:53:32.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.613+0000 7f9466d83640 1 -- 172.21.15.72:0/1420662014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 msgr2=0x7f94600b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.613+0000 7f9466d83640 1 --2- 172.21.15.72:0/1420662014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 0x7f94600b4040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94540097a0 tx=0x7f945402b8b0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.613+0000 7f9466d83640 1 -- 172.21.15.72:0/1420662014 shutdown_connections 2023-12-12T09:53:32.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.613+0000 7f9466d83640 1 --2- 172.21.15.72:0/1420662014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f946006c410 0x7f946015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.613+0000 7f9466d83640 1 --2- 172.21.15.72:0/1420662014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 0x7f94600b4040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.613+0000 7f9466d83640 1 -- 172.21.15.72:0/1420662014 >> 172.21.15.72:0/1420662014 conn(0x7f94600b0a90 msgr2=0x7f94600b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:32.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.613+0000 7f9466d83640 1 -- 172.21.15.72:0/1420662014 shutdown_connections 2023-12-12T09:53:32.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.613+0000 7f9466d83640 1 -- 172.21.15.72:0/1420662014 wait complete. 2023-12-12T09:53:32.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.616+0000 7f9466d83640 1 Processor -- start 2023-12-12T09:53:32.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.616+0000 7f9466d83640 1 -- start start 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9466d83640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946006c410 0x7f946015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9466d83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9460072180 0x7f94600cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9466d83640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 0x7f94600ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9466d83640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600cca20 con 0x7f946006c410 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9466d83640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600cccc0 con 0x7f9460143e00 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9466d83640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600ccfa0 con 0x7f9460072180 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9464b1f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946006c410 0x7f946015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9464b1f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946006c410 0x7f946015a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45034/0 (socket says 172.21.15.72:45034) 2023-12-12T09:53:32.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9464b1f640 1 -- 172.21.15.72:0/3188229340 learned_addr learned my addr 172.21.15.72:0/3188229340 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:32.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9464b1f640 1 -- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9460072180 msgr2=0x7f94600cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f945ffff640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9460072180 0x7f94600cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.618+0000 7f9465320640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 0x7f94600ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f9464b1f640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9460072180 0x7f94600cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f9464b1f640 1 -- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 msgr2=0x7f94600ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f9464b1f640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 0x7f94600ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f9464b1f640 1 -- 172.21.15.72:0/3188229340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458009b30 con 0x7f946006c410 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f945ffff640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9460072180 0x7f94600cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f9465320640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 0x7f94600ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f9464b1f640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946006c410 0x7f946015a9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9454000c00 tx=0x7f94540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f945d7fa640 1 -- 172.21.15.72:0/3188229340 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454006e10 con 0x7f946006c410 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f945d7fa640 1 -- 172.21.15.72:0/3188229340 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945403ad60 con 0x7f946006c410 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f945d7fa640 1 -- 172.21.15.72:0/3188229340 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454044630 con 0x7f946006c410 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.619+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9454009400 con 0x7f946006c410 2023-12-12T09:53:32.622 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.620+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94600cd560 con 0x7f946006c410 2023-12-12T09:53:32.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.621+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94600745d0 con 0x7f946006c410 2023-12-12T09:53:32.623 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.621+0000 7f945d7fa640 1 -- 172.21.15.72:0/3188229340 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f945404d450 con 0x7f946006c410 2023-12-12T09:53:32.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.623+0000 7f945d7fa640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f944406ae40 0x7f944406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:32.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.623+0000 7f945d7fa640 1 -- 172.21.15.72:0/3188229340 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f94540abbb0 con 0x7f946006c410 2023-12-12T09:53:32.625 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.623+0000 7f945ffff640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f944406ae40 0x7f944406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:32.626 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.624+0000 7f945ffff640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f944406ae40 0x7f944406d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f94600cb650 tx=0x7f944c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:32.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.627+0000 7f945d7fa640 1 -- 172.21.15.72:0/3188229340 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94540768a0 con 0x7f946006c410 2023-12-12T09:53:32.819 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.817+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f946006e020 con 0x7f946006c410 2023-12-12T09:53:32.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.820+0000 7f945d7fa640 1 -- 172.21.15.72:0/3188229340 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7f9454079ec0 con 0x7f946006c410 2023-12-12T09:53:32.823 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:32.823 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:32.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f944406ae40 msgr2=0x7f944406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f944406ae40 0x7f944406d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f94600cb650 tx=0x7f944c009040 comp rx=0 tx=0).stop 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946006c410 msgr2=0x7f946015a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946006c410 0x7f946015a9e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9454000c00 tx=0x7f94540046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 shutdown_connections 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f944406ae40 0x7f944406d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9460143e00 0x7f94600ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9460072180 0x7f94600cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 --2- 172.21.15.72:0/3188229340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946006c410 0x7f946015a9e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 >> 172.21.15.72:0/3188229340 conn(0x7f94600b0a90 msgr2=0x7f94600af290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:32.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.823+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 shutdown_connections 2023-12-12T09:53:32.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:32.824+0000 7f9466d83640 1 -- 172.21.15.72:0/3188229340 wait complete. 2023-12-12T09:53:32.844 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-12T09:53:32.845 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:53:33.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.052+0000 7fecc13e6640 1 Processor -- start 2023-12-12T09:53:33.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.052+0000 7fecc13e6640 1 -- start start 2023-12-12T09:53:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.053+0000 7fecc13e6640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fecbc159fd0 0x7fecbc0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.053+0000 7fecc13e6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fecbc070750 0x7fecbc070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.053+0000 7fecc13e6640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc0715f0 con 0x7fecbc0711e0 2023-12-12T09:53:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.053+0000 7fecc13e6640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc0705f0 con 0x7fecbc159fd0 2023-12-12T09:53:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.053+0000 7fecc13e6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc070070 con 0x7fecbc070750 2023-12-12T09:53:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.054+0000 7fecbbfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fecbc159fd0 0x7fecbc0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.054+0000 7fecbbfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fecbc159fd0 0x7fecbc0ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45048/0 (socket says 172.21.15.72:45048) 2023-12-12T09:53:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.054+0000 7fecbbfff640 1 -- 172.21.15.72:0/4255610956 learned_addr learned my addr 172.21.15.72:0/4255610956 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.054+0000 7fecc0be5640 1 --1- 172.21.15.72:0/4255610956 >> v1:172.21.15.72:6789/0 conn(0x7fecbc0711e0 0x7fecbc0ac700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39372/0 (socket says 172.21.15.72:39372) 2023-12-12T09:53:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.054+0000 7fecbbfff640 1 -- 172.21.15.72:0/4255610956 >> v2:172.21.15.113:3300/0 conn(0x7fecbc070750 msgr2=0x7fecbc070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.054+0000 7fecbbfff640 1 --2- 172.21.15.72:0/4255610956 >> v2:172.21.15.113:3300/0 conn(0x7fecbc070750 0x7fecbc070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.054+0000 7fecbbfff640 1 -- 172.21.15.72:0/4255610956 >> v1:172.21.15.72:6789/0 conn(0x7fecbc0711e0 legacy=0x7fecbc0ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.054+0000 7fecbbfff640 1 -- 172.21.15.72:0/4255610956 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc070310 con 0x7fecbc159fd0 2023-12-12T09:53:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.055+0000 7fecbbfff640 1 --2- 172.21.15.72:0/4255610956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc159fd0 0x7fecbc0ae1d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fecbc070000 tx=0x7fecac02ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48edab59b363a5c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecbaffd640 1 -- 172.21.15.72:0/4255610956 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac02f860 con 0x7fecbc159fd0 2023-12-12T09:53:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecbaffd640 1 -- 172.21.15.72:0/4255610956 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac02f9c0 con 0x7fecbc159fd0 2023-12-12T09:53:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecbaffd640 1 -- 172.21.15.72:0/4255610956 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac02fd50 con 0x7fecbc159fd0 2023-12-12T09:53:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecc13e6640 1 -- 172.21.15.72:0/4255610956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc159fd0 msgr2=0x7fecbc0ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecc13e6640 1 --2- 172.21.15.72:0/4255610956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc159fd0 0x7fecbc0ae1d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fecbc070000 tx=0x7fecac02ba40 comp rx=0 tx=0).stop 2023-12-12T09:53:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecc13e6640 1 -- 172.21.15.72:0/4255610956 shutdown_connections 2023-12-12T09:53:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecc13e6640 1 --2- 172.21.15.72:0/4255610956 >> v2:172.21.15.113:3300/0 conn(0x7fecbc070750 0x7fecbc070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecc13e6640 1 --2- 172.21.15.72:0/4255610956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc159fd0 0x7fecbc0ae1d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecc13e6640 1 -- 172.21.15.72:0/4255610956 >> 172.21.15.72:0/4255610956 conn(0x7fecbc0b0860 msgr2=0x7fecbc0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecc13e6640 1 -- 172.21.15.72:0/4255610956 shutdown_connections 2023-12-12T09:53:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.056+0000 7fecc13e6640 1 -- 172.21.15.72:0/4255610956 wait complete. 2023-12-12T09:53:33.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.058+0000 7fecc13e6640 1 Processor -- start 2023-12-12T09:53:33.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.058+0000 7fecc13e6640 1 -- start start 2023-12-12T09:53:33.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecc13e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecbc070750 0x7fecbc0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecc13e6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc0711e0 0x7fecbc150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecc13e6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecbc159fd0 0x7fecbc0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecc13e6640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc151160 con 0x7fecbc0711e0 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecc13e6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc159480 con 0x7fecbc070750 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecc13e6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1574f0 con 0x7fecbc159fd0 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecbb7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc0711e0 0x7fecbc150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecbb7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc0711e0 0x7fecbc150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45058/0 (socket says 172.21.15.72:45058) 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.059+0000 7fecbb7fe640 1 -- 172.21.15.72:0/2696930478 learned_addr learned my addr 172.21.15.72:0/2696930478 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecbbfff640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecbc070750 0x7fecbc0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecc0be5640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecbc159fd0 0x7fecbc0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecbb7fe640 1 -- 172.21.15.72:0/2696930478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecbc159fd0 msgr2=0x7fecbc0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecbb7fe640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecbc159fd0 0x7fecbc0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecbb7fe640 1 -- 172.21.15.72:0/2696930478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecbc070750 msgr2=0x7fecbc0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecbb7fe640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecbc070750 0x7fecbc0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecbb7fe640 1 -- 172.21.15.72:0/2696930478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0094d0 con 0x7fecbc0711e0 2023-12-12T09:53:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecbb7fe640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc0711e0 0x7fecbc150c20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b410 tx=0x7fecb000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecb8ff9640 1 -- 172.21.15.72:0/2696930478 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0011830 con 0x7fecbc0711e0 2023-12-12T09:53:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecb8ff9640 1 -- 172.21.15.72:0/2696930478 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb00119b0 con 0x7fecbc0711e0 2023-12-12T09:53:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecb8ff9640 1 -- 172.21.15.72:0/2696930478 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0011d60 con 0x7fecbc0711e0 2023-12-12T09:53:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.060+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc1577d0 con 0x7fecbc0711e0 2023-12-12T09:53:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.061+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc157d90 con 0x7fecbc0711e0 2023-12-12T09:53:33.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.062+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc074450 con 0x7fecbc0711e0 2023-12-12T09:53:33.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.062+0000 7fecb8ff9640 1 -- 172.21.15.72:0/2696930478 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecb001a400 con 0x7fecbc0711e0 2023-12-12T09:53:33.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.063+0000 7fecb8ff9640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feca806ae40 0x7feca806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.063+0000 7fecb8ff9640 1 -- 172.21.15.72:0/2696930478 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fecb0089b70 con 0x7fecbc0711e0 2023-12-12T09:53:33.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.064+0000 7fecbbfff640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feca806ae40 0x7feca806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.064+0000 7fecbbfff640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feca806ae40 0x7feca806d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fecac036000 tx=0x7fecac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:33.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.068+0000 7fecb8ff9640 1 -- 172.21.15.72:0/2696930478 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecb0054850 con 0x7fecbc0711e0 2023-12-12T09:53:33.137 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:53:33.147 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:53:33.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.301+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecbc0b0380 con 0x7fecbc0711e0 2023-12-12T09:53:33.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.302+0000 7fecb8ff9640 1 -- 172.21.15.72:0/2696930478 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fecb0015020 con 0x7fecbc0711e0 2023-12-12T09:53:33.304 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:33.305 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:35.924557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:53:33.305 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-12-12T09:53:33.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feca806ae40 msgr2=0x7feca806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feca806ae40 0x7feca806d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fecac036000 tx=0x7fecac008030 comp rx=0 tx=0).stop 2023-12-12T09:53:33.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc0711e0 msgr2=0x7fecbc150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc0711e0 0x7fecbc150c20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b410 tx=0x7fecb000b8d0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 shutdown_connections 2023-12-12T09:53:33.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feca806ae40 0x7feca806d300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecbc159fd0 0x7fecbc0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc0711e0 0x7fecbc150c20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 --2- 172.21.15.72:0/2696930478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecbc070750 0x7fecbc0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 >> 172.21.15.72:0/2696930478 conn(0x7fecbc0b0860 msgr2=0x7fecbc0ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:33.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 shutdown_connections 2023-12-12T09:53:33.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.305+0000 7fecc13e6640 1 -- 172.21.15.72:0/2696930478 wait complete. 2023-12-12T09:53:33.326 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:53:33.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.522+0000 7f6821b41640 1 Processor -- start 2023-12-12T09:53:33.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.522+0000 7f6821b41640 1 -- start start 2023-12-12T09:53:33.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.523+0000 7f6821b41640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f681c1609b0 0x7f681c0d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.523+0000 7f6821b41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c0d0230 0x7f681c15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.523+0000 7f6821b41640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c15dd60 con 0x7f681c0d1220 2023-12-12T09:53:33.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.523+0000 7f6821b41640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c160d90 con 0x7f681c1609b0 2023-12-12T09:53:33.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f6821b41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c15e040 con 0x7f681c0d0230 2023-12-12T09:53:33.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681b7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f681c1609b0 0x7f681c0d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681affd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f681c0d1220 0x7f681c0cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39382/0 (socket says 172.21.15.72:39382) 2023-12-12T09:53:33.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681affd640 1 -- 172.21.15.72:0/115093586 learned_addr learned my addr 172.21.15.72:0/115093586 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:33.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681b7fe640 1 -- 172.21.15.72:0/115093586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c0d0230 msgr2=0x7f681c15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:33.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681bfff640 1 --2- 172.21.15.72:0/115093586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c0d0230 0x7f681c15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681b7fe640 1 --2- 172.21.15.72:0/115093586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c0d0230 0x7f681c15cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681a7fc640 1 -- 172.21.15.72:0/115093586 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188426704 0 0) 0x7f681c15dd60 con 0x7f681c0d1220 2023-12-12T09:53:33.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681b7fe640 1 -- 172.21.15.72:0/115093586 >> v1:172.21.15.72:6789/0 conn(0x7f681c0d1220 legacy=0x7f681c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681b7fe640 1 -- 172.21.15.72:0/115093586 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c15e320 con 0x7f681c1609b0 2023-12-12T09:53:33.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.524+0000 7f681bfff640 1 --2- 172.21.15.72:0/115093586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c0d0230 0x7f681c15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:33.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.526+0000 7f681b7fe640 1 --2- 172.21.15.72:0/115093586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c1609b0 0x7f681c0d0ce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097a0 tx=0x7f680c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c4da357291d07d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:33.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.526+0000 7f681a7fc640 1 -- 172.21.15.72:0/115093586 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c02d040 con 0x7f681c1609b0 2023-12-12T09:53:33.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.526+0000 7f681a7fc640 1 -- 172.21.15.72:0/115093586 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c031810 con 0x7f681c1609b0 2023-12-12T09:53:33.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.526+0000 7f681a7fc640 1 -- 172.21.15.72:0/115093586 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c031ba0 con 0x7f681c1609b0 2023-12-12T09:53:33.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.526+0000 7f6821b41640 1 -- 172.21.15.72:0/115093586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c1609b0 msgr2=0x7f681c0d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.526+0000 7f6821b41640 1 --2- 172.21.15.72:0/115093586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c1609b0 0x7f681c0d0ce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097a0 tx=0x7f680c02b8b0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.527+0000 7f6821b41640 1 -- 172.21.15.72:0/115093586 shutdown_connections 2023-12-12T09:53:33.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.527+0000 7f6821b41640 1 --2- 172.21.15.72:0/115093586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c0d0230 0x7f681c15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.527+0000 7f6821b41640 1 --2- 172.21.15.72:0/115093586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c1609b0 0x7f681c0d0ce0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.527+0000 7f6821b41640 1 -- 172.21.15.72:0/115093586 >> 172.21.15.72:0/115093586 conn(0x7f681c09c450 msgr2=0x7f681c0d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:33.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.527+0000 7f6821b41640 1 -- 172.21.15.72:0/115093586 shutdown_connections 2023-12-12T09:53:33.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.527+0000 7f6821b41640 1 -- 172.21.15.72:0/115093586 wait complete. 2023-12-12T09:53:33.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.529+0000 7f6821b41640 1 Processor -- start 2023-12-12T09:53:33.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.529+0000 7f6821b41640 1 -- start start 2023-12-12T09:53:33.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f6821b41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c0d0230 0x7f681c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f6821b41640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c0d1220 0x7f681c1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f6821b41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c1609b0 0x7f681c1f0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f6821b41640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c1f3070 con 0x7f681c0d1220 2023-12-12T09:53:33.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f6821b41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c1f4030 con 0x7f681c0d0230 2023-12-12T09:53:33.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f6821b41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c1f0a70 con 0x7f681c1609b0 2023-12-12T09:53:33.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f681affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c0d1220 0x7f681c1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f681affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c0d1220 0x7f681c1f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45068/0 (socket says 172.21.15.72:45068) 2023-12-12T09:53:33.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f681affd640 1 -- 172.21.15.72:0/1237076307 learned_addr learned my addr 172.21.15.72:0/1237076307 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f681bfff640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c1609b0 0x7f681c1f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.530+0000 7f681b7fe640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c0d0230 0x7f681c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f681affd640 1 -- 172.21.15.72:0/1237076307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c1609b0 msgr2=0x7f681c1f0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f681affd640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c1609b0 0x7f681c1f0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f681affd640 1 -- 172.21.15.72:0/1237076307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c0d0230 msgr2=0x7f681c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f681affd640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c0d0230 0x7f681c0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f681affd640 1 -- 172.21.15.72:0/1237076307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c009400 con 0x7f681c0d1220 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f681affd640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c0d1220 0x7f681c1f3af0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f681000c4d0 tx=0x7f681000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f6803fff640 1 -- 172.21.15.72:0/1237076307 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68100074e0 con 0x7f681c0d1220 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f6803fff640 1 -- 172.21.15.72:0/1237076307 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6810007660 con 0x7f681c0d1220 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c0c9b30 con 0x7f681c0d1220 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f6803fff640 1 -- 172.21.15.72:0/1237076307 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810004a20 con 0x7f681c0d1220 2023-12-12T09:53:33.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.531+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c1f32f0 con 0x7f681c0d1220 2023-12-12T09:53:33.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.533+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c0745d0 con 0x7f681c0d1220 2023-12-12T09:53:33.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.534+0000 7f6803fff640 1 -- 172.21.15.72:0/1237076307 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6810004b80 con 0x7f681c0d1220 2023-12-12T09:53:33.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.535+0000 7f6803fff640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f680806ae40 0x7f680806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.535+0000 7f6803fff640 1 -- 172.21.15.72:0/1237076307 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6810088f00 con 0x7f681c0d1220 2023-12-12T09:53:33.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.535+0000 7f681b7fe640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f680806ae40 0x7f680806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.536+0000 7f681b7fe640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f680806ae40 0x7f680806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f680c036000 tx=0x7f680c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:33.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.540+0000 7f6803fff640 1 -- 172.21.15.72:0/1237076307 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6810053bf0 con 0x7f681c0d1220 2023-12-12T09:53:33.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.769+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f681c0d3820 con 0x7f681c0d1220 2023-12-12T09:53:33.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.770+0000 7f6803fff640 1 -- 172.21.15.72:0/1237076307 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f6810018020 con 0x7f681c0d1220 2023-12-12T09:53:33.772 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:33.772 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/574067809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":574067809},{"type":"v1","addr":"172.21.15.113:6834","nonce":574067809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3034056619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3034056619},{"type":"v1","addr":"172.21.15.72:6835","nonce":3034056619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2991913020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2991913020},{"type":"v1","addr":"172.21.15.113:6835","nonce":2991913020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:52:07.436771+0000","modified":"2023-12-12T09:52:35.924557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4243":{"gid":4243,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/1645740011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1645740011},{"type":"v1","addr":"172.21.15.72:6837","nonce":1645740011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:53:33.772 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.772+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f680806ae40 msgr2=0x7f680806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.772+0000 7f6821b41640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f680806ae40 0x7f680806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f680c036000 tx=0x7f680c03c040 comp rx=0 tx=0).stop 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c0d1220 msgr2=0x7f681c1f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c0d1220 0x7f681c1f3af0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f681000c4d0 tx=0x7f681000c990 comp rx=0 tx=0).stop 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 shutdown_connections 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f680806ae40 0x7f680806d300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f681c1609b0 0x7f681c1f0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f681c0d1220 0x7f681c1f3af0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 --2- 172.21.15.72:0/1237076307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c0d0230 0x7f681c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 >> 172.21.15.72:0/1237076307 conn(0x7f681c09c450 msgr2=0x7f681c06b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 shutdown_connections 2023-12-12T09:53:33.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.773+0000 7f6821b41640 1 -- 172.21.15.72:0/1237076307 wait complete. 2023-12-12T09:53:33.794 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:33.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.984+0000 7f2f35c56640 1 Processor -- start 2023-12-12T09:53:33.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.984+0000 7f2f35c56640 1 -- start start 2023-12-12T09:53:33.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f35c56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3015e2b0 0x7f2f30164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f35c56640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3006ca40 con 0x7f2f30157a60 2023-12-12T09:53:33.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f35c56640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f30071cd0 con 0x7f2f3006c410 2023-12-12T09:53:33.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f35c56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30070f10 con 0x7f2f3015e2b0 2023-12-12T09:53:33.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f2f7fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f2f30157a60 0x7f2f3006de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39386/0 (socket says 172.21.15.72:39386) 2023-12-12T09:53:33.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f2f7fe640 1 -- 172.21.15.72:0/4229048482 learned_addr learned my addr 172.21.15.72:0/4229048482 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f2ffff640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3015e2b0 0x7f2f30164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192107987 0 0) 0x7f2f3006ca40 con 0x7f2f30157a60 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.986+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f1c005730 con 0x7f2f30157a60 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3008812084 0 0) 0x7f2f1c005730 con 0x7f2f30157a60 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3015e2b0 msgr2=0x7f2f30164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3015e2b0 0x7f2f30164d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 >> v1:172.21.15.113:6789/0 conn(0x7f2f3006c410 legacy=0x7f2f3015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f300711f0 con 0x7f2f30157a60 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f2f20001670 con 0x7f2f30157a60 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 >> v1:172.21.15.72:6789/0 conn(0x7f2f30157a60 legacy=0x7f2f3006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f1c005bb0 0x7f2f1c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f1c0069d0 0x7f2f1c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3015e2b0 0x7f2f1c007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30070f10 con 0x7f2f1c005bb0 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c00dd50 con 0x7f2f1c0069d0 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c00df10 con 0x7f2f3015e2b0 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.987+0000 7f2f2f7fe640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f1c005bb0 0x7f2f1c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.988+0000 7f2f2ffff640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3015e2b0 0x7f2f1c007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.988+0000 7f2f2f7fe640 1 -- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3015e2b0 msgr2=0x7f2f1c007440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.988+0000 7f2f2f7fe640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3015e2b0 0x7f2f1c007440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.988+0000 7f2f2f7fe640 1 -- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f1c0069d0 msgr2=0x7f2f1c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.988+0000 7f2f2f7fe640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f1c0069d0 0x7f2f1c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.988+0000 7f2f2f7fe640 1 -- 172.21.15.72:0/4229048482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c005730 con 0x7f2f1c005bb0 2023-12-12T09:53:33.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.989+0000 7f2f2f7fe640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f1c005bb0 0x7f2f1c005fd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c0e0 tx=0x7f2f1c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fc6b34fdd3c27b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:33.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.989+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20005ad0 con 0x7f2f1c005bb0 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.989+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f20005c30 con 0x7f2f1c005bb0 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.989+0000 7f2f2effd640 1 -- 172.21.15.72:0/4229048482 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2002c810 con 0x7f2f1c005bb0 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.989+0000 7f2f35c56640 1 -- 172.21.15.72:0/4229048482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f1c005bb0 msgr2=0x7f2f1c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.989+0000 7f2f35c56640 1 --2- 172.21.15.72:0/4229048482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f1c005bb0 0x7f2f1c005fd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c0e0 tx=0x7f2f1c007a00 comp rx=0 tx=0).stop 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.989+0000 7f2f2ffff640 1 -- 172.21.15.72:0/4229048482 reap_dead start 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.990+0000 7f2f35c56640 1 -- 172.21.15.72:0/4229048482 shutdown_connections 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.990+0000 7f2f35c56640 1 -- 172.21.15.72:0/4229048482 >> 172.21.15.72:0/4229048482 conn(0x7f2f300b0a90 msgr2=0x7f2f300b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.990+0000 7f2f35c56640 1 -- 172.21.15.72:0/4229048482 shutdown_connections 2023-12-12T09:53:33.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.990+0000 7f2f35c56640 1 -- 172.21.15.72:0/4229048482 wait complete. 2023-12-12T09:53:33.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.991+0000 7f2f35c56640 1 Processor -- start 2023-12-12T09:53:33.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.991+0000 7f2f35c56640 1 -- start start 2023-12-12T09:53:33.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.992+0000 7f2f35c56640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f1c005bb0 0x7f2f300a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.992+0000 7f2f35c56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f30147c30 0x7f2f301edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.992+0000 7f2f35c56640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f30141f90 0x7f2f30142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:33.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.992+0000 7f2f35c56640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301f26a0 con 0x7f2f30141f90 2023-12-12T09:53:33.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.992+0000 7f2f35c56640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301441c0 con 0x7f2f1c005bb0 2023-12-12T09:53:33.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.992+0000 7f2f35c56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301f2bd0 con 0x7f2f30147c30 2023-12-12T09:53:33.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f30141f90 0x7f2f30142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:33.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f30141f90 0x7f2f30142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45082/0 (socket says 172.21.15.72:45082) 2023-12-12T09:53:33.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 -- 172.21.15.72:0/1568395416 learned_addr learned my addr 172.21.15.72:0/1568395416 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:34.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2f7fe640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f1c005bb0 0x7f2f300a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 -- 172.21.15.72:0/1568395416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f30147c30 msgr2=0x7f2f301edfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f30147c30 0x7f2f301edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 -- 172.21.15.72:0/1568395416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f1c005bb0 msgr2=0x7f2f300a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f1c005bb0 0x7f2f300a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 -- 172.21.15.72:0/1568395416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f300a8990 con 0x7f2f30141f90 2023-12-12T09:53:34.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2ffff640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f30141f90 0x7f2f30142440 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2f28007e90 tx=0x7f2f2800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.993+0000 7f2f2cff9640 1 -- 172.21.15.72:0/1568395416 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28017400 con 0x7f2f30141f90 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.994+0000 7f2f2cff9640 1 -- 172.21.15.72:0/1568395416 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28011040 con 0x7f2f30141f90 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.994+0000 7f2f2cff9640 1 -- 172.21.15.72:0/1568395416 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280177a0 con 0x7f2f30141f90 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.994+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f30154ff0 con 0x7f2f30141f90 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.994+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f30154ce0 con 0x7f2f30141f90 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.995+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f300745d0 con 0x7f2f30141f90 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.996+0000 7f2f2cff9640 1 -- 172.21.15.72:0/1568395416 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f280203b0 con 0x7f2f30141f90 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.996+0000 7f2f2cff9640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2f1006ae70 0x7f2f1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.996+0000 7f2f2cff9640 1 -- 172.21.15.72:0/1568395416 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2f28089c20 con 0x7f2f30141f90 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.997+0000 7f2f2f7fe640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2f1006ae70 0x7f2f1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:33.998+0000 7f2f2f7fe640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2f1006ae70 0x7f2f1006d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f20034000 tx=0x7f2f20009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:34.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.001+0000 7f2f2cff9640 1 -- 172.21.15.72:0/1568395416 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f28054870 con 0x7f2f30141f90 2023-12-12T09:53:34.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.193+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f3006d4b0 con 0x7f2f30141f90 2023-12-12T09:53:34.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.194+0000 7f2f2cff9640 1 -- 172.21.15.72:0/1568395416 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18735 (secure 0 0 0) 0x7f2f28057e90 con 0x7f2f30141f90 2023-12-12T09:53:34.198 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:34.199 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:52:55.418759+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:52:05.675969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:52:06.464147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:52:10.083525+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:34.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2f1006ae70 msgr2=0x7f2f1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2f1006ae70 0x7f2f1006d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f20034000 tx=0x7f2f20009030 comp rx=0 tx=0).stop 2023-12-12T09:53:34.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f30141f90 msgr2=0x7f2f30142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f30141f90 0x7f2f30142440 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2f28007e90 tx=0x7f2f2800f4a0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 shutdown_connections 2023-12-12T09:53:34.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2f1006ae70 0x7f2f1006d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f30141f90 0x7f2f30142440 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f30147c30 0x7f2f301edfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 --2- 172.21.15.72:0/1568395416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f1c005bb0 0x7f2f300a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 >> 172.21.15.72:0/1568395416 conn(0x7f2f300b0a90 msgr2=0x7f2f301560d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:34.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 shutdown_connections 2023-12-12T09:53:34.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.197+0000 7f2f35c56640 1 -- 172.21.15.72:0/1568395416 wait complete. 2023-12-12T09:53:34.217 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-12T09:53:34.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.419+0000 7ff0cebe6640 1 Processor -- start 2023-12-12T09:53:34.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.419+0000 7ff0cebe6640 1 -- start start 2023-12-12T09:53:34.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.420+0000 7ff0cebe6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c81437b0 0x7ff0c80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.421+0000 7ff0cebe6640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c815e700 con 0x7ff0c806a460 2023-12-12T09:53:34.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.421+0000 7ff0cebe6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c8143b90 con 0x7ff0c815dd80 2023-12-12T09:53:34.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.421+0000 7ff0cebe6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80d0170 con 0x7ff0c81437b0 2023-12-12T09:53:34.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.421+0000 7ff0cd3e3640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff0c806a460 0x7ff0c815d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39392/0 (socket says 172.21.15.72:39392) 2023-12-12T09:53:34.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.421+0000 7ff0cd3e3640 1 -- 172.21.15.72:0/2501506447 learned_addr learned my addr 172.21.15.72:0/2501506447 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:34.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.421+0000 7ff0cdbe4640 1 --2- 172.21.15.72:0/2501506447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c81437b0 0x7ff0c80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.422+0000 7ff0cdbe4640 1 -- 172.21.15.72:0/2501506447 >> v1:172.21.15.113:6789/0 conn(0x7ff0c815dd80 legacy=0x7ff0c806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.422+0000 7ff0cdbe4640 1 -- 172.21.15.72:0/2501506447 >> v1:172.21.15.72:6789/0 conn(0x7ff0c806a460 legacy=0x7ff0c815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.422+0000 7ff0cdbe4640 1 -- 172.21.15.72:0/2501506447 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c815e420 con 0x7ff0c81437b0 2023-12-12T09:53:34.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.423+0000 7ff0cdbe4640 1 --2- 172.21.15.72:0/2501506447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c81437b0 0x7ff0c80d0590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097d0 tx=0x7ff0c402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d6ee25d7feaf66 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:34.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.425+0000 7ff0ccbe2640 1 -- 172.21.15.72:0/2501506447 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4031800 con 0x7ff0c81437b0 2023-12-12T09:53:34.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.425+0000 7ff0ccbe2640 1 -- 172.21.15.72:0/2501506447 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4031960 con 0x7ff0c81437b0 2023-12-12T09:53:34.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.425+0000 7ff0ccbe2640 1 -- 172.21.15.72:0/2501506447 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4031cf0 con 0x7ff0c81437b0 2023-12-12T09:53:34.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.426+0000 7ff0cebe6640 1 -- 172.21.15.72:0/2501506447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c81437b0 msgr2=0x7ff0c80d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.426+0000 7ff0cebe6640 1 --2- 172.21.15.72:0/2501506447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c81437b0 0x7ff0c80d0590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097d0 tx=0x7ff0c402b8a0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.426+0000 7ff0cebe6640 1 -- 172.21.15.72:0/2501506447 shutdown_connections 2023-12-12T09:53:34.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.426+0000 7ff0cebe6640 1 --2- 172.21.15.72:0/2501506447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c81437b0 0x7ff0c80d0590 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.426+0000 7ff0cebe6640 1 -- 172.21.15.72:0/2501506447 >> 172.21.15.72:0/2501506447 conn(0x7ff0c806b570 msgr2=0x7ff0c806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:34.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.426+0000 7ff0cebe6640 1 -- 172.21.15.72:0/2501506447 shutdown_connections 2023-12-12T09:53:34.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.426+0000 7ff0cebe6640 1 -- 172.21.15.72:0/2501506447 wait complete. 2023-12-12T09:53:34.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.427+0000 7ff0cebe6640 1 Processor -- start 2023-12-12T09:53:34.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.427+0000 7ff0cebe6640 1 -- start start 2023-12-12T09:53:34.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.428+0000 7ff0cebe6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0c806a460 0x7ff0c80c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.428+0000 7ff0cebe6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0c81437b0 0x7ff0c813ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.428+0000 7ff0cebe6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c815dd80 0x7ff0c815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.428+0000 7ff0cebe6640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c813e230 con 0x7ff0c806a460 2023-12-12T09:53:34.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.428+0000 7ff0cebe6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c813d3a0 con 0x7ff0c81437b0 2023-12-12T09:53:34.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.428+0000 7ff0cebe6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c813d640 con 0x7ff0c815dd80 2023-12-12T09:53:34.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0c806a460 0x7ff0c80c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0c806a460 0x7ff0c80c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45092/0 (socket says 172.21.15.72:45092) 2023-12-12T09:53:34.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 -- 172.21.15.72:0/3369725165 learned_addr learned my addr 172.21.15.72:0/3369725165 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 -- 172.21.15.72:0/3369725165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c815dd80 msgr2=0x7ff0c815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c815dd80 0x7ff0c815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 -- 172.21.15.72:0/3369725165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0c81437b0 msgr2=0x7ff0c813ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0c81437b0 0x7ff0c813ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 -- 172.21.15.72:0/3369725165 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4009400 con 0x7ff0c806a460 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.429+0000 7ff0cdbe4640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0c806a460 0x7ff0c80c4df0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40098f0 tx=0x7ff0c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.430+0000 7ff0be7fc640 1 -- 172.21.15.72:0/3369725165 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c403bcc0 con 0x7ff0c806a460 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.430+0000 7ff0be7fc640 1 -- 172.21.15.72:0/3369725165 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c403be60 con 0x7ff0c806a460 2023-12-12T09:53:34.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.430+0000 7ff0be7fc640 1 -- 172.21.15.72:0/3369725165 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4045630 con 0x7ff0c806a460 2023-12-12T09:53:34.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.430+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c813d920 con 0x7ff0c806a460 2023-12-12T09:53:34.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.430+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c815cc40 con 0x7ff0c806a460 2023-12-12T09:53:34.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.431+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c813e430 con 0x7ff0c806a460 2023-12-12T09:53:34.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.432+0000 7ff0be7fc640 1 -- 172.21.15.72:0/3369725165 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0c40443a0 con 0x7ff0c806a460 2023-12-12T09:53:34.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.433+0000 7ff0be7fc640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff0ac06ae40 0x7ff0ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.433+0000 7ff0be7fc640 1 -- 172.21.15.72:0/3369725165 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff0c402e070 con 0x7ff0c806a460 2023-12-12T09:53:34.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.434+0000 7ff0cd3e3640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff0ac06ae40 0x7ff0ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.434+0000 7ff0cd3e3640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff0ac06ae40 0x7ff0ac06d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8143180 tx=0x7ff0b8007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:34.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.438+0000 7ff0be7fc640 1 -- 172.21.15.72:0/3369725165 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0c4076ce0 con 0x7ff0c806a460 2023-12-12T09:53:34.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.665+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff0c8074450 con 0x7ff0c806a460 2023-12-12T09:53:34.667 INFO:tasks.ceph.mon.a.smithi072.stderr:2023-12-12T09:53:34.666+0000 7f3fa4ad2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-12T09:53:34.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.669+0000 7ff0be7fc640 1 -- 172.21.15.72:0/3369725165 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff0c4035090 con 0x7ff0c806a460 2023-12-12T09:53:34.671 INFO:teuthology.orchestra.run.smithi072.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-12T09:53:34.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff0ac06ae40 msgr2=0x7ff0ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff0ac06ae40 0x7ff0ac06d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8143180 tx=0x7ff0b8007a40 comp rx=0 tx=0).stop 2023-12-12T09:53:34.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0c806a460 msgr2=0x7ff0c80c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0c806a460 0x7ff0c80c4df0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40098f0 tx=0x7ff0c40046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 shutdown_connections 2023-12-12T09:53:34.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff0ac06ae40 0x7ff0ac06d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0c815dd80 0x7ff0c815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0c81437b0 0x7ff0c813ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 --2- 172.21.15.72:0/3369725165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0c806a460 0x7ff0c80c4df0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 >> 172.21.15.72:0/3369725165 conn(0x7ff0c806b570 msgr2=0x7ff0c806d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:34.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 shutdown_connections 2023-12-12T09:53:34.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.673+0000 7ff0cebe6640 1 -- 172.21.15.72:0/3369725165 wait complete. 2023-12-12T09:53:34.686 INFO:tasks.ceph.mds.c.smithi072.stderr: -5977> 2023-12-12T09:52:11.819+0000 7f43d3e15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-12T09:53:34.692 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-12T09:53:34.740 INFO:tasks.ceph.mds.c.smithi072.stdout:starting mds.c at 2023-12-12T09:53:34.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.866+0000 7fa123347640 1 Processor -- start 2023-12-12T09:53:34.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.866+0000 7fa123347640 1 -- start start 2023-12-12T09:53:34.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.867+0000 7fa123347640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0ce150 0x7fa11c0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.867+0000 7fa123347640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa11c0cedf0 0x7fa11c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.867+0000 7fa123347640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c06c500 con 0x7fa11c155400 2023-12-12T09:53:34.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.868+0000 7fa123347640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0cf7c0 con 0x7fa11c0cedf0 2023-12-12T09:53:34.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.868+0000 7fa123347640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c15b4a0 con 0x7fa11c0ce150 2023-12-12T09:53:34.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.868+0000 7fa122b46640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa11c0cedf0 0x7fa11c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.868+0000 7fa121b44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0ce150 0x7fa11c0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.868+0000 7fa121b44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0ce150 0x7fa11c0ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:48702/0 (socket says 172.21.15.72:48702) 2023-12-12T09:53:34.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.868+0000 7fa121b44640 1 -- 172.21.15.72:0/1574751339 learned_addr learned my addr 172.21.15.72:0/1574751339 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.869+0000 7fa121b44640 1 -- 172.21.15.72:0/1574751339 >> v2:172.21.15.113:3300/0 conn(0x7fa11c0cedf0 msgr2=0x7fa11c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.869+0000 7fa121b44640 1 --2- 172.21.15.72:0/1574751339 >> v2:172.21.15.113:3300/0 conn(0x7fa11c0cedf0 0x7fa11c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.869+0000 7fa121b44640 1 -- 172.21.15.72:0/1574751339 >> v1:172.21.15.113:6789/0 conn(0x7fa11c155400 legacy=0x7fa11c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.869+0000 7fa121b44640 1 -- 172.21.15.72:0/1574751339 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0adbb0 con 0x7fa11c0ce150 2023-12-12T09:53:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.869+0000 7fa122b46640 1 --2- 172.21.15.72:0/1574751339 >> v2:172.21.15.113:3300/0 conn(0x7fa11c0cedf0 0x7fa11c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:53:34.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.871+0000 7fa121b44640 1 --2- 172.21.15.72:0/1574751339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0ce150 0x7fa11c0ce5b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009870 tx=0x7fa10c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46834e429301f5a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:34.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.871+0000 7fa121343640 1 -- 172.21.15.72:0/1574751339 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c030840 con 0x7fa11c0ce150 2023-12-12T09:53:34.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.871+0000 7fa121343640 1 -- 172.21.15.72:0/1574751339 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c0309a0 con 0x7fa11c0ce150 2023-12-12T09:53:34.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.871+0000 7fa121343640 1 -- 172.21.15.72:0/1574751339 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c030d30 con 0x7fa11c0ce150 2023-12-12T09:53:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.872+0000 7fa123347640 1 -- 172.21.15.72:0/1574751339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0ce150 msgr2=0x7fa11c0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.872+0000 7fa123347640 1 --2- 172.21.15.72:0/1574751339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0ce150 0x7fa11c0ce5b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009870 tx=0x7fa10c02b980 comp rx=0 tx=0).stop 2023-12-12T09:53:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.872+0000 7fa123347640 1 -- 172.21.15.72:0/1574751339 shutdown_connections 2023-12-12T09:53:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.872+0000 7fa123347640 1 --2- 172.21.15.72:0/1574751339 >> v2:172.21.15.113:3300/0 conn(0x7fa11c0cedf0 0x7fa11c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.872+0000 7fa123347640 1 --2- 172.21.15.72:0/1574751339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0ce150 0x7fa11c0ce5b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.872+0000 7fa123347640 1 -- 172.21.15.72:0/1574751339 >> 172.21.15.72:0/1574751339 conn(0x7fa11c06a640 msgr2=0x7fa11c06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.872+0000 7fa123347640 1 -- 172.21.15.72:0/1574751339 shutdown_connections 2023-12-12T09:53:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.872+0000 7fa123347640 1 -- 172.21.15.72:0/1574751339 wait complete. 2023-12-12T09:53:34.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.874+0000 7fa123347640 1 Processor -- start 2023-12-12T09:53:34.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.874+0000 7fa123347640 1 -- start start 2023-12-12T09:53:34.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.875+0000 7fa123347640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa11c0ce150 0x7fa11c149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.875+0000 7fa123347640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0cedf0 0x7fa11c14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.875+0000 7fa123347640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa11c155400 0x7fa11c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa123347640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c14c7f0 con 0x7fa11c0ce150 2023-12-12T09:53:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa123347640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c14ea50 con 0x7fa11c155400 2023-12-12T09:53:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa123347640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c14e490 con 0x7fa11c0cedf0 2023-12-12T09:53:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122345640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa11c0ce150 0x7fa11c149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122345640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa11c0ce150 0x7fa11c149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45118/0 (socket says 172.21.15.72:45118) 2023-12-12T09:53:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122345640 1 -- 172.21.15.72:0/652148885 learned_addr learned my addr 172.21.15.72:0/652148885 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122345640 1 -- 172.21.15.72:0/652148885 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0cedf0 msgr2=0x7fa11c14d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122345640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0cedf0 0x7fa11c14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122b46640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa11c155400 0x7fa11c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122345640 1 -- 172.21.15.72:0/652148885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa11c155400 msgr2=0x7fa11c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122345640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa11c155400 0x7fa11c15d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.876+0000 7fa122345640 1 -- 172.21.15.72:0/652148885 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114009a70 con 0x7fa11c0ce150 2023-12-12T09:53:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.877+0000 7fa122345640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa11c0ce150 0x7fa11c149a00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c400 tx=0x7fa11800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.877+0000 7fa112ffd640 1 -- 172.21.15.72:0/652148885 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180052d0 con 0x7fa11c0ce150 2023-12-12T09:53:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.877+0000 7fa112ffd640 1 -- 172.21.15.72:0/652148885 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa118005430 con 0x7fa11c0ce150 2023-12-12T09:53:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.877+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c0094d0 con 0x7fa11c0ce150 2023-12-12T09:53:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.877+0000 7fa112ffd640 1 -- 172.21.15.72:0/652148885 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118003830 con 0x7fa11c0ce150 2023-12-12T09:53:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.877+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c14ca20 con 0x7fa11c0ce150 2023-12-12T09:53:34.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.878+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c074450 con 0x7fa11c0ce150 2023-12-12T09:53:34.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.879+0000 7fa112ffd640 1 -- 172.21.15.72:0/652148885 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa118003990 con 0x7fa11c0ce150 2023-12-12T09:53:34.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.880+0000 7fa112ffd640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa10006ae40 0x7fa10006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:34.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.880+0000 7fa112ffd640 1 -- 172.21.15.72:0/652148885 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fa118089040 con 0x7fa11c0ce150 2023-12-12T09:53:34.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.880+0000 7fa121b44640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa10006ae40 0x7fa10006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:34.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.881+0000 7fa121b44640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa10006ae40 0x7fa10006d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa10c036000 tx=0x7fa10c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:34.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:34.885+0000 7fa112ffd640 1 -- 172.21.15.72:0/652148885 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1180bbe90 con 0x7fa11c0ce150 2023-12-12T09:53:35.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.112+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa11c06d750 con 0x7fa11c0ce150 2023-12-12T09:53:35.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.672+0000 7fa112ffd640 1 -- 172.21.15.72:0/652148885 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa118053c40 con 0x7fa11c0ce150 2023-12-12T09:53:35.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa10006ae40 msgr2=0x7fa10006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:35.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa10006ae40 0x7fa10006d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa10c036000 tx=0x7fa10c03b040 comp rx=0 tx=0).stop 2023-12-12T09:53:35.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa11c0ce150 msgr2=0x7fa11c149a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:35.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa11c0ce150 0x7fa11c149a00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c400 tx=0x7fa11800c8c0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 shutdown_connections 2023-12-12T09:53:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa10006ae40 0x7fa10006d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa11c155400 0x7fa11c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa11c0cedf0 0x7fa11c14d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 --2- 172.21.15.72:0/652148885 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa11c0ce150 0x7fa11c149a00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 >> 172.21.15.72:0/652148885 conn(0x7fa11c06a640 msgr2=0x7fa11c0b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 shutdown_connections 2023-12-12T09:53:35.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.676+0000 7fa123347640 1 -- 172.21.15.72:0/652148885 wait complete. 2023-12-12T09:53:35.699 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-12T09:53:35.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.905+0000 7ff6be583640 1 Processor -- start 2023-12-12T09:53:35.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.905+0000 7ff6be583640 1 -- start start 2023-12-12T09:53:35.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.907+0000 7ff6be583640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff6b80b3770 0x7ff6b80cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:35.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.907+0000 7ff6be583640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b815c4b0 con 0x7ff6b80ce1b0 2023-12-12T09:53:35.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.907+0000 7ff6be583640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b80cb700 con 0x7ff6b80cf6c0 2023-12-12T09:53:35.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.907+0000 7ff6be583640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b80cd750 con 0x7ff6b80b3770 2023-12-12T09:53:35.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.907+0000 7ff6bd581640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff6b80b3770 0x7ff6b80cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:35.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.907+0000 7ff6bcd80640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff6b80ce1b0 0x7ff6b80cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59328/0 (socket says 172.21.15.72:59328) 2023-12-12T09:53:35.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.907+0000 7ff6bcd80640 1 -- 172.21.15.72:0/709913104 learned_addr learned my addr 172.21.15.72:0/709913104 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:35.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.908+0000 7ff6bd581640 1 -- 172.21.15.72:0/709913104 >> v1:172.21.15.113:6789/0 conn(0x7ff6b80cf6c0 legacy=0x7ff6b80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:35.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.908+0000 7ff6bd581640 1 -- 172.21.15.72:0/709913104 >> v1:172.21.15.72:6789/0 conn(0x7ff6b80ce1b0 legacy=0x7ff6b80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:35.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.908+0000 7ff6bd581640 1 -- 172.21.15.72:0/709913104 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b80b3b50 con 0x7ff6b80b3770 2023-12-12T09:53:35.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.908+0000 7ff6affff640 1 -- 172.21.15.72:0/709913104 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722857239 0 0) 0x7ff6a8001110 con 0x7ff6b80ce1b0 2023-12-12T09:53:35.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.909+0000 7ff6bd581640 1 --2- 172.21.15.72:0/709913104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80cb1c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40097a0 tx=0x7ff6b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540f1925863cd229 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:35.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.909+0000 7ff6affff640 1 -- 172.21.15.72:0/709913104 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4031830 con 0x7ff6b80b3770 2023-12-12T09:53:35.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.909+0000 7ff6affff640 1 -- 172.21.15.72:0/709913104 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b4031990 con 0x7ff6b80b3770 2023-12-12T09:53:35.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.909+0000 7ff6affff640 1 -- 172.21.15.72:0/709913104 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4031d20 con 0x7ff6b80b3770 2023-12-12T09:53:35.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.910+0000 7ff6be583640 1 -- 172.21.15.72:0/709913104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 msgr2=0x7ff6b80cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:35.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.910+0000 7ff6be583640 1 --2- 172.21.15.72:0/709913104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80cb1c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40097a0 tx=0x7ff6b402b8d0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.910+0000 7ff6be583640 1 -- 172.21.15.72:0/709913104 shutdown_connections 2023-12-12T09:53:35.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.910+0000 7ff6be583640 1 --2- 172.21.15.72:0/709913104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80cb1c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.910+0000 7ff6be583640 1 -- 172.21.15.72:0/709913104 >> 172.21.15.72:0/709913104 conn(0x7ff6b806a880 msgr2=0x7ff6b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:35.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.910+0000 7ff6be583640 1 -- 172.21.15.72:0/709913104 shutdown_connections 2023-12-12T09:53:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.910+0000 7ff6be583640 1 -- 172.21.15.72:0/709913104 wait complete. 2023-12-12T09:53:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.912+0000 7ff6be583640 1 Processor -- start 2023-12-12T09:53:35.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.912+0000 7ff6be583640 1 -- start start 2023-12-12T09:53:35.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6be583640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:35.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6be583640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6b80ce1b0 0x7ff6b80705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:35.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6be583640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6b80cf6c0 0x7ff6b8157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:35.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6be583640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8072110 con 0x7ff6b80b3770 2023-12-12T09:53:35.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6be583640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8071c80 con 0x7ff6b80cf6c0 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6be583640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b8071070 con 0x7ff6b80ce1b0 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6bd581640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6bd581640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52760/0 (socket says 172.21.15.72:52760) 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.913+0000 7ff6bd581640 1 -- 172.21.15.72:0/3624975474 learned_addr learned my addr 172.21.15.72:0/3624975474 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6bd581640 1 -- 172.21.15.72:0/3624975474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6b80ce1b0 msgr2=0x7ff6b80705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6bd581640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6b80ce1b0 0x7ff6b80705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6bd581640 1 -- 172.21.15.72:0/3624975474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6b80cf6c0 msgr2=0x7ff6b8157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6bd581640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6b80cf6c0 0x7ff6b8157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:35.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6bd581640 1 -- 172.21.15.72:0/3624975474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b4009400 con 0x7ff6b80b3770 2023-12-12T09:53:35.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6bd581640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80b2930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40097a0 tx=0x7ff6b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:35.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6adffb640 1 -- 172.21.15.72:0/3624975474 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b403aa60 con 0x7ff6b80b3770 2023-12-12T09:53:35.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6adffb640 1 -- 172.21.15.72:0/3624975474 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b403abc0 con 0x7ff6b80b3770 2023-12-12T09:53:35.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6adffb640 1 -- 172.21.15.72:0/3624975474 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40443f0 con 0x7ff6b80b3770 2023-12-12T09:53:35.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b8071350 con 0x7ff6b80b3770 2023-12-12T09:53:35.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.914+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b8071910 con 0x7ff6b80b3770 2023-12-12T09:53:35.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.915+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b8074450 con 0x7ff6b80b3770 2023-12-12T09:53:35.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.916+0000 7ff6adffb640 1 -- 172.21.15.72:0/3624975474 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6b404d450 con 0x7ff6b80b3770 2023-12-12T09:53:35.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.917+0000 7ff6adffb640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff69c06ae40 0x7ff69c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:35.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.917+0000 7ff6adffb640 1 -- 172.21.15.72:0/3624975474 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6749+0+0 (secure 0 0 0) 0x7ff6b40abcf0 con 0x7ff6b80b3770 2023-12-12T09:53:35.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.918+0000 7ff6bcd80640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff69c06ae40 0x7ff69c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:35.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.918+0000 7ff6bcd80640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff69c06ae40 0x7ff69c06d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8007fd0 tx=0x7ff6a8001310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:35.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:35.922+0000 7ff6adffb640 1 -- 172.21.15.72:0/3624975474 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6b40768f0 con 0x7ff6b80b3770 2023-12-12T09:53:36.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.104+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff6b8073340 con 0x7ff6b80b3770 2023-12-12T09:53:36.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.684+0000 7ff6adffb640 1 -- 172.21.15.72:0/3624975474 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff6b4079f10 con 0x7ff6b80b3770 2023-12-12T09:53:36.687 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' removed 2023-12-12T09:53:36.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.689+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff69c06ae40 msgr2=0x7ff69c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:36.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.689+0000 7ff6be583640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff69c06ae40 0x7ff69c06d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8007fd0 tx=0x7ff6a8001310 comp rx=0 tx=0).stop 2023-12-12T09:53:36.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.689+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 msgr2=0x7ff6b80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:36.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.689+0000 7ff6be583640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80b2930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40097a0 tx=0x7ff6b40046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.691+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 shutdown_connections 2023-12-12T09:53:36.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.691+0000 7ff6be583640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff69c06ae40 0x7ff69c06d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.691+0000 7ff6be583640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6b80cf6c0 0x7ff6b8157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.691+0000 7ff6be583640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6b80ce1b0 0x7ff6b80705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.691+0000 7ff6be583640 1 --2- 172.21.15.72:0/3624975474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6b80b3770 0x7ff6b80b2930 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.691+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 >> 172.21.15.72:0/3624975474 conn(0x7ff6b806a880 msgr2=0x7ff6b80af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:36.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.691+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 shutdown_connections 2023-12-12T09:53:36.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.692+0000 7ff6be583640 1 -- 172.21.15.72:0/3624975474 wait complete. 2023-12-12T09:53:36.715 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-12T09:53:36.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.926+0000 7f4dda92c640 1 Processor -- start 2023-12-12T09:53:36.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.926+0000 7f4dda92c640 1 -- start start 2023-12-12T09:53:36.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.928+0000 7f4dda92c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4dd406e4e0 0x7f4dd415e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:36.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.928+0000 7f4dda92c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd4071d90 0x7f4dd4072210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.928+0000 7f4dda92c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd415e6a0 con 0x7f4dd406d540 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.928+0000 7f4dda92c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd4157c90 con 0x7f4dd406e4e0 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.928+0000 7f4dda92c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40cf9d0 con 0x7f4dd4071d90 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.928+0000 7f4dd3fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4dd406e4e0 0x7f4dd415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.928+0000 7f4dd3fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4dd406e4e0 0x7f4dd415e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52774/0 (socket says 172.21.15.72:52774) 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.928+0000 7f4dd3fff640 1 -- 172.21.15.72:0/2617565945 learned_addr learned my addr 172.21.15.72:0/2617565945 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.929+0000 7f4dd3fff640 1 -- 172.21.15.72:0/2617565945 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd4071d90 msgr2=0x7f4dd4072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.929+0000 7f4dd3fff640 1 --2- 172.21.15.72:0/2617565945 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd4071d90 0x7f4dd4072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.929+0000 7f4dd3fff640 1 -- 172.21.15.72:0/2617565945 >> v1:172.21.15.113:6789/0 conn(0x7f4dd406d540 legacy=0x7f4dd406a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.929+0000 7f4dd3fff640 1 -- 172.21.15.72:0/2617565945 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40cddc0 con 0x7f4dd406e4e0 2023-12-12T09:53:36.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.930+0000 7f4dd3fff640 1 --2- 172.21.15.72:0/2617565945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd406e4e0 0x7f4dd415e160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009870 tx=0x7f4dc402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf07d9d0dcbe4e44 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:36.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.930+0000 7f4dd2ffd640 1 -- 172.21.15.72:0/2617565945 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4031640 con 0x7f4dd406e4e0 2023-12-12T09:53:36.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.930+0000 7f4dd2ffd640 1 -- 172.21.15.72:0/2617565945 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc40317a0 con 0x7f4dd406e4e0 2023-12-12T09:53:36.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.930+0000 7f4dd2ffd640 1 -- 172.21.15.72:0/2617565945 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4031b30 con 0x7f4dd406e4e0 2023-12-12T09:53:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.931+0000 7f4dda92c640 1 -- 172.21.15.72:0/2617565945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd406e4e0 msgr2=0x7f4dd415e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.931+0000 7f4dda92c640 1 --2- 172.21.15.72:0/2617565945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd406e4e0 0x7f4dd415e160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009870 tx=0x7f4dc402b8f0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.931+0000 7f4dda92c640 1 -- 172.21.15.72:0/2617565945 shutdown_connections 2023-12-12T09:53:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.931+0000 7f4dda92c640 1 --2- 172.21.15.72:0/2617565945 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd4071d90 0x7f4dd4072210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.931+0000 7f4dda92c640 1 --2- 172.21.15.72:0/2617565945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd406e4e0 0x7f4dd415e160 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.931+0000 7f4dda92c640 1 -- 172.21.15.72:0/2617565945 >> 172.21.15.72:0/2617565945 conn(0x7f4dd40cfe30 msgr2=0x7f4dd40d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.931+0000 7f4dda92c640 1 -- 172.21.15.72:0/2617565945 shutdown_connections 2023-12-12T09:53:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.931+0000 7f4dda92c640 1 -- 172.21.15.72:0/2617565945 wait complete. 2023-12-12T09:53:36.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.933+0000 7f4dda92c640 1 Processor -- start 2023-12-12T09:53:36.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.933+0000 7f4dda92c640 1 -- start start 2023-12-12T09:53:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.934+0000 7f4dda92c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd406d540 0x7f4dd41538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.934+0000 7f4dda92c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dd406e4e0 0x7f4dd414ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.934+0000 7f4dda92c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd4071d90 0x7f4dd414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.934+0000 7f4dda92c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40b3d60 con 0x7f4dd4071d90 2023-12-12T09:53:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.934+0000 7f4dda92c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40b3450 con 0x7f4dd406e4e0 2023-12-12T09:53:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.934+0000 7f4dda92c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4150700 con 0x7f4dd406d540 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.934+0000 7f4dd8ec9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd4071d90 0x7f4dd414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.934+0000 7f4dd8ec9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd4071d90 0x7f4dd414fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52780/0 (socket says 172.21.15.72:52780) 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd8ec9640 1 -- 172.21.15.72:0/367665861 learned_addr learned my addr 172.21.15.72:0/367665861 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd3fff640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd406d540 0x7f4dd41538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd37fe640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dd406e4e0 0x7f4dd414ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd8ec9640 1 -- 172.21.15.72:0/367665861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd406d540 msgr2=0x7f4dd41538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd8ec9640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd406d540 0x7f4dd41538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd8ec9640 1 -- 172.21.15.72:0/367665861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dd406e4e0 msgr2=0x7f4dd414ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd8ec9640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dd406e4e0 0x7f4dd414ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:36.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd8ec9640 1 -- 172.21.15.72:0/367665861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc40094d0 con 0x7f4dd4071d90 2023-12-12T09:53:36.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd8ec9640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd4071d90 0x7f4dd414fe10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00b990 tx=0x7f4dcc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:36.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.935+0000 7f4dd0ff9640 1 -- 172.21.15.72:0/367665861 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00cba0 con 0x7f4dd4071d90 2023-12-12T09:53:36.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.936+0000 7f4dd0ff9640 1 -- 172.21.15.72:0/367665861 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc00cd00 con 0x7f4dd4071d90 2023-12-12T09:53:36.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.936+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd40b36a0 con 0x7f4dd4071d90 2023-12-12T09:53:36.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.936+0000 7f4dd0ff9640 1 -- 172.21.15.72:0/367665861 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc014680 con 0x7f4dd4071d90 2023-12-12T09:53:36.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.936+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd414cf60 con 0x7f4dd4071d90 2023-12-12T09:53:36.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.937+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd4067910 con 0x7f4dd4071d90 2023-12-12T09:53:36.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.938+0000 7f4dd0ff9640 1 -- 172.21.15.72:0/367665861 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4dcc0147e0 con 0x7f4dd4071d90 2023-12-12T09:53:36.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.939+0000 7f4dd0ff9640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4dc006ae40 0x7f4dc006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:36.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.939+0000 7f4dd0ff9640 1 -- 172.21.15.72:0/367665861 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f4dcc0897c0 con 0x7f4dd4071d90 2023-12-12T09:53:36.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.939+0000 7f4dd3fff640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4dc006ae40 0x7f4dc006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:36.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.939+0000 7f4dd3fff640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4dc006ae40 0x7f4dc006d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4036000 tx=0x7f4dc403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:36.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:36.943+0000 7f4dd0ff9640 1 -- 172.21.15.72:0/367665861 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dcc054600 con 0x7f4dd4071d90 2023-12-12T09:53:37.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.134+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4dd40b3ea0 con 0x7f4dd4071d90 2023-12-12T09:53:37.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.695+0000 7f4dd0ff9640 1 -- 172.21.15.72:0/367665861 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4dd40b3ea0 con 0x7f4dd4071d90 2023-12-12T09:53:37.697 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' removed 2023-12-12T09:53:37.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4dc006ae40 msgr2=0x7f4dc006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:37.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4dc006ae40 0x7f4dc006d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4036000 tx=0x7f4dc403c040 comp rx=0 tx=0).stop 2023-12-12T09:53:37.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd4071d90 msgr2=0x7f4dd414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:37.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd4071d90 0x7f4dd414fe10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00b990 tx=0x7f4dcc00be50 comp rx=0 tx=0).stop 2023-12-12T09:53:37.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 shutdown_connections 2023-12-12T09:53:37.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4dc006ae40 0x7f4dc006d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4dd4071d90 0x7f4dd414fe10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4dd406e4e0 0x7f4dd414ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 --2- 172.21.15.72:0/367665861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4dd406d540 0x7f4dd41538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.700+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 >> 172.21.15.72:0/367665861 conn(0x7f4dd40cfe30 msgr2=0x7f4dd406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:37.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.702+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 shutdown_connections 2023-12-12T09:53:37.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.702+0000 7f4dda92c640 1 -- 172.21.15.72:0/367665861 wait complete. 2023-12-12T09:53:37.727 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-12-12T09:53:37.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.926+0000 7f3b87fff640 1 Processor -- start 2023-12-12T09:53:37.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.926+0000 7f3b87fff640 1 -- start start 2023-12-12T09:53:37.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.928+0000 7f3b87fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b880d15b0 0x7f3b880d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:37.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.928+0000 7f3b87fff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8815d8b0 con 0x7f3b8815d4a0 2023-12-12T09:53:37.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.928+0000 7f3b87fff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b880ce5d0 con 0x7f3b880d1990 2023-12-12T09:53:37.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.928+0000 7f3b87fff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b880ce870 con 0x7f3b880d15b0 2023-12-12T09:53:37.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.928+0000 7f3b877fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f3b8815d4a0 0x7f3b880afdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59330/0 (socket says 172.21.15.72:59330) 2023-12-12T09:53:37.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.928+0000 7f3b877fe640 1 -- 172.21.15.72:0/2666024398 learned_addr learned my addr 172.21.15.72:0/2666024398 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:37.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b86ffd640 1 --2- 172.21.15.72:0/2666024398 >> v2:172.21.15.113:3300/0 conn(0x7f3b880d15b0 0x7f3b880d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:37.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704181974 0 0) 0x7f3b8815d8b0 con 0x7f3b8815d4a0 2023-12-12T09:53:37.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b70003520 con 0x7f3b8815d4a0 2023-12-12T09:53:37.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4261316826 0 0) 0x7f3b70003520 con 0x7f3b8815d4a0 2023-12-12T09:53:37.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 >> v2:172.21.15.113:3300/0 conn(0x7f3b880d15b0 msgr2=0x7f3b880d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:37.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 --2- 172.21.15.72:0/2666024398 >> v2:172.21.15.113:3300/0 conn(0x7f3b880d15b0 0x7f3b880d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 >> v1:172.21.15.113:6789/0 conn(0x7f3b880d1990 legacy=0x7f3b8815cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b880ceb50 con 0x7f3b8815d4a0 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f3b800019f0 con 0x7f3b8815d4a0 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 >> v1:172.21.15.72:6789/0 conn(0x7f3b8815d4a0 legacy=0x7f3b880afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70003970 0x7f3b70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b700047c0 0x7f3b70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b70005180 0x7f3b7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880ce870 con 0x7f3b70005180 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7000c120 con 0x7f3b70003970 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.929+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7000c2e0 con 0x7f3b700047c0 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.930+0000 7f3b877fe640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b70005180 0x7f3b7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.930+0000 7f3b86ffd640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70003970 0x7f3b70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.930+0000 7f3b867fc640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b700047c0 0x7f3b70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:37.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.930+0000 7f3b877fe640 1 -- 172.21.15.72:0/2666024398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b700047c0 msgr2=0x7f3b70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:37.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.930+0000 7f3b877fe640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b700047c0 0x7f3b70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.930+0000 7f3b877fe640 1 -- 172.21.15.72:0/2666024398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70003970 msgr2=0x7f3b70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:37.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.930+0000 7f3b877fe640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70003970 0x7f3b70003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.930+0000 7f3b877fe640 1 -- 172.21.15.72:0/2666024398 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70003520 con 0x7f3b70005180 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.931+0000 7f3b877fe640 1 -- 172.21.15.72:0/2666024398 reap_dead start 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.931+0000 7f3b877fe640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b70005180 0x7f3b7000b980 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000a830 tx=0x7f3b700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d2e1733854a485 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.931+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800072d0 con 0x7f3b70005180 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8002f070 con 0x7f3b70005180 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b85ffb640 1 -- 172.21.15.72:0/2666024398 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800076b0 con 0x7f3b70005180 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b87fff640 1 -- 172.21.15.72:0/2666024398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b70005180 msgr2=0x7f3b7000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b87fff640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b70005180 0x7f3b7000b980 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000a830 tx=0x7f3b700057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b87fff640 1 -- 172.21.15.72:0/2666024398 shutdown_connections 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b87fff640 1 --2- 172.21.15.72:0/2666024398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b70005180 0x7f3b7000b980 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b87fff640 1 -- 172.21.15.72:0/2666024398 >> 172.21.15.72:0/2666024398 conn(0x7f3b880d2f10 msgr2=0x7f3b880d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:37.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b87fff640 1 -- 172.21.15.72:0/2666024398 shutdown_connections 2023-12-12T09:53:37.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.932+0000 7f3b87fff640 1 -- 172.21.15.72:0/2666024398 wait complete. 2023-12-12T09:53:37.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.934+0000 7f3b87fff640 1 Processor -- start 2023-12-12T09:53:37.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.934+0000 7f3b87fff640 1 -- start start 2023-12-12T09:53:37.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.935+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70005180 0x7f3b881533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:37.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.935+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b8809cc10 0x7f3b8809d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:37.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.935+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b880cb0c0 0x7f3b880cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:37.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.935+0000 7f3b87fff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8809d5b0 con 0x7f3b880cb0c0 2023-12-12T09:53:37.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.935+0000 7f3b87fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880cbe80 con 0x7f3b70005180 2023-12-12T09:53:37.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.935+0000 7f3b87fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880cc160 con 0x7f3b8809cc10 2023-12-12T09:53:37.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b880cb0c0 0x7f3b880cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:37.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b880cb0c0 0x7f3b880cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52788/0 (socket says 172.21.15.72:52788) 2023-12-12T09:53:37.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 -- 172.21.15.72:0/1544897276 learned_addr learned my addr 172.21.15.72:0/1544897276 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b86ffd640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70005180 0x7f3b881533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b867fc640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b8809cc10 0x7f3b8809d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 -- 172.21.15.72:0/1544897276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b8809cc10 msgr2=0x7f3b8809d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b8809cc10 0x7f3b8809d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 -- 172.21.15.72:0/1544897276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70005180 msgr2=0x7f3b881533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70005180 0x7f3b881533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 -- 172.21.15.72:0/1544897276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b880cc440 con 0x7f3b880cb0c0 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.936+0000 7f3b877fe640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b880cb0c0 0x7f3b880cb540 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b8002cc40 tx=0x7f3b80001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.937+0000 7f3b6f7fe640 1 -- 172.21.15.72:0/1544897276 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80044070 con 0x7f3b880cb0c0 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.937+0000 7f3b6f7fe640 1 -- 172.21.15.72:0/1544897276 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b80001250 con 0x7f3b880cb0c0 2023-12-12T09:53:37.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.937+0000 7f3b6f7fe640 1 -- 172.21.15.72:0/1544897276 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800075c0 con 0x7f3b880cb0c0 2023-12-12T09:53:37.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.937+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b880cc720 con 0x7f3b880cb0c0 2023-12-12T09:53:37.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.937+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b881538f0 con 0x7f3b880cb0c0 2023-12-12T09:53:37.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.938+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b88074450 con 0x7f3b880cb0c0 2023-12-12T09:53:37.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.939+0000 7f3b6f7fe640 1 -- 172.21.15.72:0/1544897276 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b80031030 con 0x7f3b880cb0c0 2023-12-12T09:53:37.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.940+0000 7f3b6f7fe640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3b7406ae70 0x7f3b7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:37.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.940+0000 7f3b6f7fe640 1 -- 172.21.15.72:0/1544897276 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f3b800aafd0 con 0x7f3b880cb0c0 2023-12-12T09:53:37.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.940+0000 7f3b86ffd640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3b7406ae70 0x7f3b7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:37.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.941+0000 7f3b86ffd640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3b7406ae70 0x7f3b7406d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009990 tx=0x7f3b78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:37.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:37.944+0000 7f3b6f7fe640 1 -- 172.21.15.72:0/1544897276 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b80075fa0 con 0x7f3b880cb0c0 2023-12-12T09:53:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.133+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3b88074450 con 0x7f3b880cb0c0 2023-12-12T09:53:38.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.699+0000 7f3b6f7fe640 1 -- 172.21.15.72:0/1544897276 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f3b800795c0 con 0x7f3b880cb0c0 2023-12-12T09:53:38.701 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data_ec' removed 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.703+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3b7406ae70 msgr2=0x7f3b7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.703+0000 7f3b87fff640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3b7406ae70 0x7f3b7406d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009990 tx=0x7f3b78008040 comp rx=0 tx=0).stop 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.703+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b880cb0c0 msgr2=0x7f3b880cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.703+0000 7f3b87fff640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b880cb0c0 0x7f3b880cb540 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b8002cc40 tx=0x7f3b80001d80 comp rx=0 tx=0).stop 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.704+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 shutdown_connections 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.704+0000 7f3b87fff640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3b7406ae70 0x7f3b7406d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.704+0000 7f3b87fff640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b880cb0c0 0x7f3b880cb540 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.704+0000 7f3b87fff640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b8809cc10 0x7f3b8809d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.704+0000 7f3b87fff640 1 --2- 172.21.15.72:0/1544897276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b70005180 0x7f3b881533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.704+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 >> 172.21.15.72:0/1544897276 conn(0x7f3b880d2f10 msgr2=0x7f3b880b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.705+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 shutdown_connections 2023-12-12T09:53:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.705+0000 7f3b87fff640 1 -- 172.21.15.72:0/1544897276 wait complete. 2023-12-12T09:53:38.730 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-12T09:53:38.730 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-12T09:53:38.730 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-12T09:53:38.730 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:53:38.731 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-12T09:53:38.731 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-12T09:53:38.731 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-12T09:53:38.731 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:53:38.731 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-12T09:53:38.731 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-12T09:53:38.731 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-12T09:53:38.732 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:53:38.732 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-12T09:53:38.732 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-12T09:53:38.732 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-12T09:53:38.732 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:53:38.747 INFO:tasks.ceph.mds.c:Stopped 2023-12-12T09:53:38.748 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-12T09:53:38.749 INFO:tasks.ceph.mds.a:Stopped 2023-12-12T09:53:38.749 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-12T09:53:38.753 INFO:tasks.ceph.mds.d:Stopped 2023-12-12T09:53:38.753 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-12T09:53:38.756 INFO:tasks.ceph.mds.b:Stopped 2023-12-12T09:53:38.756 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-12T09:53:38.797 INFO:tasks.ceph.mds.c:Started 2023-12-12T09:53:38.797 DEBUG:teuthology.parallel:result is None 2023-12-12T09:53:38.804 INFO:tasks.ceph.mds.d:Started 2023-12-12T09:53:38.805 DEBUG:teuthology.parallel:result is None 2023-12-12T09:53:38.805 INFO:tasks.ceph.mds.a:Started 2023-12-12T09:53:38.805 DEBUG:teuthology.parallel:result is None 2023-12-12T09:53:38.812 INFO:tasks.ceph.mds.b:Started 2023-12-12T09:53:38.812 DEBUG:teuthology.parallel:result is None 2023-12-12T09:53:38.812 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-12T09:53:38.955 INFO:tasks.ceph.mds.c.smithi072.stdout:starting mds.c at 2023-12-12T09:53:38.956 INFO:tasks.ceph.mds.a.smithi072.stdout:starting mds.a at 2023-12-12T09:53:38.966 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-12-12T09:53:38.967 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-12-12T09:53:38.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.996+0000 7fccd8305640 1 Processor -- start 2023-12-12T09:53:38.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.997+0000 7fccd8305640 1 -- start start 2023-12-12T09:53:39.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd8305640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fccd015f260 0x7fccd00ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd8305640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd01626b0 0x7fccd00701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd8305640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd0070b60 con 0x7fccd0070700 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd8305640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd00cd040 con 0x7fccd015f260 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd8305640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd015f640 con 0x7fccd01626b0 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd68a2640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fccd0070700 0x7fccd006c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59362/0 (socket says 172.21.15.72:59362) 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd68a2640 1 -- 172.21.15.72:0/3528738889 learned_addr learned my addr 172.21.15.72:0/3528738889 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd58a0640 1 --2- 172.21.15.72:0/3528738889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd01626b0 0x7fccd00701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.998+0000 7fccd60a1640 1 --2- 172.21.15.72:0/3528738889 >> v2:172.21.15.113:3300/0 conn(0x7fccd015f260 0x7fccd00ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.999+0000 7fccd509f640 1 -- 172.21.15.72:0/3528738889 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8795050 0 0) 0x7fccd0070b60 con 0x7fccd0070700 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.999+0000 7fccd509f640 1 -- 172.21.15.72:0/3528738889 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccbc003520 con 0x7fccd0070700 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.999+0000 7fccd58a0640 1 -- 172.21.15.72:0/3528738889 >> v2:172.21.15.113:3300/0 conn(0x7fccd015f260 msgr2=0x7fccd00ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.999+0000 7fccd58a0640 1 --2- 172.21.15.72:0/3528738889 >> v2:172.21.15.113:3300/0 conn(0x7fccd015f260 0x7fccd00ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.999+0000 7fccd58a0640 1 -- 172.21.15.72:0/3528738889 >> v1:172.21.15.72:6789/0 conn(0x7fccd0070700 legacy=0x7fccd006c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:38.999+0000 7fccd58a0640 1 -- 172.21.15.72:0/3528738889 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00cde40 con 0x7fccd01626b0 2023-12-12T09:53:39.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.000+0000 7fccd58a0640 1 --2- 172.21.15.72:0/3528738889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd01626b0 0x7fccd00701c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fccc00097a0 tx=0x7fccc002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272e2b44737d3aca server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:39.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.000+0000 7fccd509f640 1 -- 172.21.15.72:0/3528738889 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc002d040 con 0x7fccd01626b0 2023-12-12T09:53:39.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.000+0000 7fccd509f640 1 -- 172.21.15.72:0/3528738889 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc0031810 con 0x7fccd01626b0 2023-12-12T09:53:39.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.000+0000 7fccd509f640 1 -- 172.21.15.72:0/3528738889 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0031b80 con 0x7fccd01626b0 2023-12-12T09:53:39.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.001+0000 7fccd8305640 1 -- 172.21.15.72:0/3528738889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd01626b0 msgr2=0x7fccd00701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.001+0000 7fccd8305640 1 --2- 172.21.15.72:0/3528738889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd01626b0 0x7fccd00701c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fccc00097a0 tx=0x7fccc002b8b0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.001+0000 7fccd8305640 1 -- 172.21.15.72:0/3528738889 shutdown_connections 2023-12-12T09:53:39.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.001+0000 7fccd8305640 1 --2- 172.21.15.72:0/3528738889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd01626b0 0x7fccd00701c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.001+0000 7fccd8305640 1 --2- 172.21.15.72:0/3528738889 >> v2:172.21.15.113:3300/0 conn(0x7fccd015f260 0x7fccd00ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.001+0000 7fccd8305640 1 -- 172.21.15.72:0/3528738889 >> 172.21.15.72:0/3528738889 conn(0x7fccd00b0a90 msgr2=0x7fccd00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:39.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.001+0000 7fccd8305640 1 -- 172.21.15.72:0/3528738889 shutdown_connections 2023-12-12T09:53:39.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.001+0000 7fccd8305640 1 -- 172.21.15.72:0/3528738889 wait complete. 2023-12-12T09:53:39.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.003+0000 7fccd8305640 1 Processor -- start 2023-12-12T09:53:39.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.003+0000 7fccd8305640 1 -- start start 2023-12-12T09:53:39.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd8305640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccd0070700 0x7fccd00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd8305640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd015f260 0x7fccd00a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd8305640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccd01626b0 0x7fccd015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd8305640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00b19b0 con 0x7fccd01626b0 2023-12-12T09:53:39.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd8305640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00a8ea0 con 0x7fccd0070700 2023-12-12T09:53:39.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd8305640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00a9180 con 0x7fccd015f260 2023-12-12T09:53:39.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd68a2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccd01626b0 0x7fccd015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd68a2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccd01626b0 0x7fccd015ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52822/0 (socket says 172.21.15.72:52822) 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd68a2640 1 -- 172.21.15.72:0/2309196552 learned_addr learned my addr 172.21.15.72:0/2309196552 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd58a0640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd015f260 0x7fccd00a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd60a1640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccd0070700 0x7fccd00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd68a2640 1 -- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd015f260 msgr2=0x7fccd00a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd68a2640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd015f260 0x7fccd00a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd68a2640 1 -- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccd0070700 msgr2=0x7fccd00b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd68a2640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccd0070700 0x7fccd00b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd68a2640 1 -- 172.21.15.72:0/2309196552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0009400 con 0x7fccd01626b0 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.004+0000 7fccd58a0640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd015f260 0x7fccd00a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.005+0000 7fccd68a2640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccd01626b0 0x7fccd015ac80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fccc800cb40 tx=0x7fccc800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.005+0000 7fccc6ffd640 1 -- 172.21.15.72:0/2309196552 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800aca0 con 0x7fccd01626b0 2023-12-12T09:53:39.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.005+0000 7fccc6ffd640 1 -- 172.21.15.72:0/2309196552 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc800ae00 con 0x7fccd01626b0 2023-12-12T09:53:39.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.005+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd00a9460 con 0x7fccd01626b0 2023-12-12T09:53:39.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.005+0000 7fccc6ffd640 1 -- 172.21.15.72:0/2309196552 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8012620 con 0x7fccd01626b0 2023-12-12T09:53:39.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.005+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd0160ea0 con 0x7fccd01626b0 2023-12-12T09:53:39.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.006+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd00b1e40 con 0x7fccd01626b0 2023-12-12T09:53:39.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.007+0000 7fccc6ffd640 1 -- 172.21.15.72:0/2309196552 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fccc8007070 con 0x7fccd01626b0 2023-12-12T09:53:39.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.008+0000 7fccc6ffd640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fccb406ae40 0x7fccb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.008+0000 7fccc6ffd640 1 -- 172.21.15.72:0/2309196552 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fccc8089540 con 0x7fccd01626b0 2023-12-12T09:53:39.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.008+0000 7fccd60a1640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fccb406ae40 0x7fccb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.009+0000 7fccd60a1640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fccb406ae40 0x7fccb406d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009ad0 tx=0x7fcccc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:39.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.013+0000 7fccc6ffd640 1 -- 172.21.15.72:0/2309196552 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fccc8054710 con 0x7fccd01626b0 2023-12-12T09:53:39.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.216+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fccd00745d0 con 0x7fccd01626b0 2023-12-12T09:53:39.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.732+0000 7fccc6ffd640 1 -- 172.21.15.72:0/2309196552 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fccc8057d30 con 0x7fccd01626b0 2023-12-12T09:53:39.734 INFO:teuthology.orchestra.run.smithi072.stderr: removed all blocklist entries 2023-12-12T09:53:39.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.735+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fccb406ae40 msgr2=0x7fccb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.735+0000 7fccd8305640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fccb406ae40 0x7fccb406d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009ad0 tx=0x7fcccc008040 comp rx=0 tx=0).stop 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.735+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccd01626b0 msgr2=0x7fccd015ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.735+0000 7fccd8305640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccd01626b0 0x7fccd015ac80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fccc800cb40 tx=0x7fccc800a060 comp rx=0 tx=0).stop 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.736+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 shutdown_connections 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.736+0000 7fccd8305640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fccb406ae40 0x7fccb406d300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.736+0000 7fccd8305640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccd01626b0 0x7fccd015ac80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.736+0000 7fccd8305640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccd015f260 0x7fccd00a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.736+0000 7fccd8305640 1 --2- 172.21.15.72:0/2309196552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccd0070700 0x7fccd00b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.736+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 >> 172.21.15.72:0/2309196552 conn(0x7fccd00b0a90 msgr2=0x7fccd015e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:39.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.736+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 shutdown_connections 2023-12-12T09:53:39.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.736+0000 7fccd8305640 1 -- 172.21.15.72:0/2309196552 wait complete. 2023-12-12T09:53:39.763 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-12T09:53:39.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.956+0000 7f1325961640 1 Processor -- start 2023-12-12T09:53:39.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.957+0000 7f1325961640 1 -- start start 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.958+0000 7f1325961640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1320143e00 0x7f13200b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f1325961640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13200b3bb0 con 0x7f1320072180 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f1325961640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13201441e0 con 0x7f132006c410 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f1325961640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1320072680 con 0x7f1320143e00 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f131effd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1320143e00 0x7f13200b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f131effd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1320143e00 0x7f13200b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52836/0 (socket says 172.21.15.72:52836) 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f131effd640 1 -- 172.21.15.72:0/3094788196 learned_addr learned my addr 172.21.15.72:0/3094788196 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f13167fc640 1 --1- 172.21.15.72:0/3094788196 >> v1:172.21.15.72:6789/0 conn(0x7f1320072180 0x7f13200b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59372/0 (socket says 172.21.15.72:59372) 2023-12-12T09:53:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f131effd640 1 -- 172.21.15.72:0/3094788196 >> v1:172.21.15.113:6789/0 conn(0x7f132006c410 legacy=0x7f132015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:39.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f131effd640 1 -- 172.21.15.72:0/3094788196 >> v1:172.21.15.72:6789/0 conn(0x7f1320072180 legacy=0x7f13200b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f131effd640 1 -- 172.21.15.72:0/3094788196 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13200b1e30 con 0x7f1320143e00 2023-12-12T09:53:39.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.959+0000 7f131e7fc640 1 -- 172.21.15.72:0/3094788196 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127699535 0 0) 0x7f13200b3bb0 con 0x7f1320072180 2023-12-12T09:53:39.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.960+0000 7f131effd640 1 --2- 172.21.15.72:0/3094788196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f13200b4040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1310009870 tx=0x7f131002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65a952f7089260f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:39.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f131e7fc640 1 -- 172.21.15.72:0/3094788196 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310030860 con 0x7f1320143e00 2023-12-12T09:53:39.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f131e7fc640 1 -- 172.21.15.72:0/3094788196 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100309c0 con 0x7f1320143e00 2023-12-12T09:53:39.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f131e7fc640 1 -- 172.21.15.72:0/3094788196 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310030d50 con 0x7f1320143e00 2023-12-12T09:53:39.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f1325961640 1 -- 172.21.15.72:0/3094788196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 msgr2=0x7f13200b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f1325961640 1 --2- 172.21.15.72:0/3094788196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f13200b4040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1310009870 tx=0x7f131002b9a0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f1325961640 1 -- 172.21.15.72:0/3094788196 shutdown_connections 2023-12-12T09:53:39.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f1325961640 1 --2- 172.21.15.72:0/3094788196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f13200b4040 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f1325961640 1 -- 172.21.15.72:0/3094788196 >> 172.21.15.72:0/3094788196 conn(0x7f13200b0a90 msgr2=0x7f13200b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:39.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.961+0000 7f1325961640 1 -- 172.21.15.72:0/3094788196 shutdown_connections 2023-12-12T09:53:39.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.962+0000 7f1325961640 1 -- 172.21.15.72:0/3094788196 wait complete. 2023-12-12T09:53:39.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.963+0000 7f1325961640 1 Processor -- start 2023-12-12T09:53:39.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.963+0000 7f1325961640 1 -- start start 2023-12-12T09:53:39.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.964+0000 7f1325961640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f132006c410 0x7f1320136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.964+0000 7f1325961640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320072180 0x7f132013aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.964+0000 7f1325961640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f1320141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.964+0000 7f1325961640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132014c4d0 con 0x7f1320143e00 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.964+0000 7f1325961640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13201374e0 con 0x7f132006c410 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.964+0000 7f1325961640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132013d800 con 0x7f1320072180 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f1320141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f1320141e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52848/0 (socket says 172.21.15.72:52848) 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 -- 172.21.15.72:0/1877503056 learned_addr learned my addr 172.21.15.72:0/1877503056 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131effd640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f132006c410 0x7f1320136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f13167fc640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320072180 0x7f132013aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 -- 172.21.15.72:0/1877503056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320072180 msgr2=0x7f132013aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320072180 0x7f132013aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 -- 172.21.15.72:0/1877503056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f132006c410 msgr2=0x7f1320136fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f132006c410 0x7f1320136fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 -- 172.21.15.72:0/1877503056 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100094d0 con 0x7f1320143e00 2023-12-12T09:53:39.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.965+0000 7f131f7fe640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f1320141e30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f131800c7b0 tx=0x7f131800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:39.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.966+0000 7f131cff9640 1 -- 172.21.15.72:0/1877503056 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318003d80 con 0x7f1320143e00 2023-12-12T09:53:39.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.966+0000 7f131cff9640 1 -- 172.21.15.72:0/1877503056 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1318005070 con 0x7f1320143e00 2023-12-12T09:53:39.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.966+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132013dae0 con 0x7f1320143e00 2023-12-12T09:53:39.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.966+0000 7f131cff9640 1 -- 172.21.15.72:0/1877503056 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318005420 con 0x7f1320143e00 2023-12-12T09:53:39.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.966+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132014c720 con 0x7f1320143e00 2023-12-12T09:53:39.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.967+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132013e000 con 0x7f1320143e00 2023-12-12T09:53:39.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.968+0000 7f131cff9640 1 -- 172.21.15.72:0/1877503056 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1318005580 con 0x7f1320143e00 2023-12-12T09:53:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.969+0000 7f131cff9640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f130006ae40 0x7f130006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.969+0000 7f131cff9640 1 -- 172.21.15.72:0/1877503056 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1318088970 con 0x7f1320143e00 2023-12-12T09:53:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.969+0000 7f131effd640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f130006ae40 0x7f130006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.970+0000 7f131effd640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f130006ae40 0x7f130006d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1310036000 tx=0x7f131003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:39.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:39.974+0000 7f131cff9640 1 -- 172.21.15.72:0/1877503056 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1318053b90 con 0x7f1320143e00 2023-12-12T09:53:40.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.233+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f132014c940 con 0x7f1320143e00 2023-12-12T09:53:40.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.235+0000 7f131cff9640 1 -- 172.21.15.72:0/1877503056 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f1318017020 con 0x7f1320143e00 2023-12-12T09:53:40.236 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: "auth_dump": [ 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mds.a", 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAKLXhlvoC3MRAAoAmTetbcBYxKsaGaD6tAIg==", 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.237 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mds.b", 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQALLXhlEGCfFxAAmpAYcdOKIMI4Wg2GyD2V1w==", 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mds.c", 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQALLXhlkWvDBhAAKV9ZAou23rMrzJJ4GURxGw==", 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:53:40.238 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mds.d", 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQALLXhlJcbVKBAAPs/uN737wS83OIndY0JCZQ==", 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.0", 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAQLXhldS0YHhAANAsQberqgVrbCB5TcgrmsA==", 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.1", 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQATLXhlyVgFKxAA3xZtCVmkuMrqrZCWloqoMA==", 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:53:40.240 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.2", 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAWLXhlqrpdMxAAmfxGvPquBX/kyiGCi/rjQg==", 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.3", 2023-12-12T09:53:40.241 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAaLXhl5nXjAhAACmfr78GlROpqJ7UKUwer7A==", 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.4", 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAhLXhlLaouGhAAqURr0qTsHKQby0Th8coLCA==", 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.5", 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAkLXhlGVwmJRAAX0ujEA/JpOrqD9N9mRR1hg==", 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.6", 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAnLXhl5VkLLxAAY8lCytX5JEfHFPZ3ABewnw==", 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:53:40.243 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.7", 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQArLXhlLJaxDBAAGd8VZfX+Ofg80KrvflSEcA==", 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.0", 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQALLXhlQjDrNhAAcu4UiqhTEuide2bwToXy+Q==", 2023-12-12T09:53:40.244 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow r", 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow rw", 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow rwx" 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.1", 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAMLXhlMLcyEBAAihZy2JhlSzOc1JbYT37pIw==", 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow r", 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow rw", 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow rwx" 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.2", 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAMLXhlPlybBRAAVk0eZQW2HZk9y6TwvEWIkQ==", 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow r", 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow rw", 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow rwx" 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.3", 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAMLXhlEkM2GxAAvsUAfS00lFnfoviP3gE/mg==", 2023-12-12T09:53:40.246 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow r", 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow rw", 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow rwx" 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.admin", 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAJLXhlz9ilJBAALwHSd+zMRvGM2rWbTxqGFw==", 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow *", 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.247 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.bootstrap-mds", 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQA0LXhlD664FxAAFVJOxw7KKp7jx6erxoZYYA==", 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile bootstrap-mds" 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.bootstrap-mgr", 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQA0LXhlk7e4FxAA+2MhWtyx0Jjaa2ZAP0hmuA==", 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.248 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.bootstrap-osd", 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQA0LXhlHMC4FxAAGZEu9SKLb5vGhXoa80TohQ==", 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile bootstrap-osd" 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.bootstrap-rbd", 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQA0LXhlosi4FxAAuPZFJdGQNv4/uuxSGZKKdw==", 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.249 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQA0LXhlGdG4FxAAmlJZGpnEQBTxZgMYm6cqdw==", 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.bootstrap-rgw", 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQA0LXhlyNm4FxAAtHaNvXvK72sn/V5esxMPAQ==", 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mgr.x", 2023-12-12T09:53:40.250 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAKLXhlujKiJRAACswJ4xyotHStzLDuXHlGPA==", 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow *", 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile mgr", 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mgr.y", 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAKLXhlmRkdGRAAV7LEG0mSAIVyOfeB2+L1fQ==", 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow *", 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile mgr", 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.251 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:40.252 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2023-12-12T09:53:40.252 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f130006ae40 msgr2=0x7f130006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f130006ae40 0x7f130006d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1310036000 tx=0x7f131003b040 comp rx=0 tx=0).stop 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 msgr2=0x7f1320141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f1320141e30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f131800c7b0 tx=0x7f131800cc70 comp rx=0 tx=0).stop 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 shutdown_connections 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f130006ae40 0x7f130006d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1320143e00 0x7f1320141e30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320072180 0x7f132013aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 --2- 172.21.15.72:0/1877503056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f132006c410 0x7f1320136fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.237+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 >> 172.21.15.72:0/1877503056 conn(0x7f13200b0a90 msgr2=0x7f1320070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.238+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 shutdown_connections 2023-12-12T09:53:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.238+0000 7f1325961640 1 -- 172.21.15.72:0/1877503056 wait complete. 2023-12-12T09:53:40.257 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-12T09:53:40.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.471+0000 7f4c826aa640 1 Processor -- start 2023-12-12T09:53:40.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.471+0000 7f4c826aa640 1 -- start start 2023-12-12T09:53:40.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.473+0000 7f4c826aa640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4c7c15e5d0 0x7f4c7c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.473+0000 7f4c826aa640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c071f90 con 0x7f4c7c071260 2023-12-12T09:53:40.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.473+0000 7f4c826aa640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c072270 con 0x7f4c7c0ae200 2023-12-12T09:53:40.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.473+0000 7f4c826aa640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c070070 con 0x7f4c7c15e5d0 2023-12-12T09:53:40.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.473+0000 7f4c816a8640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4c7c15e5d0 0x7f4c7c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:40.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.473+0000 7f4c81ea9640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4c7c071260 0x7f4c7c0cded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59380/0 (socket says 172.21.15.72:59380) 2023-12-12T09:53:40.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.473+0000 7f4c81ea9640 1 -- 172.21.15.72:0/4033881875 learned_addr learned my addr 172.21.15.72:0/4033881875 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:40.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.474+0000 7f4c816a8640 1 -- 172.21.15.72:0/4033881875 >> v1:172.21.15.113:6789/0 conn(0x7f4c7c0ae200 legacy=0x7f4c7c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:40.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.474+0000 7f4c816a8640 1 -- 172.21.15.72:0/4033881875 >> v1:172.21.15.72:6789/0 conn(0x7f4c7c071260 legacy=0x7f4c7c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.474+0000 7f4c816a8640 1 -- 172.21.15.72:0/4033881875 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c070350 con 0x7f4c7c15e5d0 2023-12-12T09:53:40.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.475+0000 7f4c816a8640 1 --2- 172.21.15.72:0/4033881875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c15e5d0 0x7f4c7c06bdf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4c780097a0 tx=0x7f4c7802b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7aae0bc0899722 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:40.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.475+0000 7f4c73fff640 1 -- 172.21.15.72:0/4033881875 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78030710 con 0x7f4c7c15e5d0 2023-12-12T09:53:40.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.475+0000 7f4c73fff640 1 -- 172.21.15.72:0/4033881875 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7802e050 con 0x7f4c7c15e5d0 2023-12-12T09:53:40.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.475+0000 7f4c73fff640 1 -- 172.21.15.72:0/4033881875 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78030a90 con 0x7f4c7c15e5d0 2023-12-12T09:53:40.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.476+0000 7f4c826aa640 1 -- 172.21.15.72:0/4033881875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c15e5d0 msgr2=0x7f4c7c06bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.476+0000 7f4c826aa640 1 --2- 172.21.15.72:0/4033881875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c15e5d0 0x7f4c7c06bdf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4c780097a0 tx=0x7f4c7802b680 comp rx=0 tx=0).stop 2023-12-12T09:53:40.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.476+0000 7f4c826aa640 1 -- 172.21.15.72:0/4033881875 shutdown_connections 2023-12-12T09:53:40.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.476+0000 7f4c826aa640 1 --2- 172.21.15.72:0/4033881875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c15e5d0 0x7f4c7c06bdf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.476+0000 7f4c826aa640 1 -- 172.21.15.72:0/4033881875 >> 172.21.15.72:0/4033881875 conn(0x7f4c7c0b0860 msgr2=0x7f4c7c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:40.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.476+0000 7f4c826aa640 1 -- 172.21.15.72:0/4033881875 shutdown_connections 2023-12-12T09:53:40.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.476+0000 7f4c826aa640 1 -- 172.21.15.72:0/4033881875 wait complete. 2023-12-12T09:53:40.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.478+0000 7f4c826aa640 1 Processor -- start 2023-12-12T09:53:40.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.478+0000 7f4c826aa640 1 -- start start 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c826aa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7c071260 0x7f4c7c13f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c826aa640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c0ae200 0x7f4c7c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c826aa640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c7c15e5d0 0x7f4c7c09af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c826aa640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c14ab80 con 0x7f4c7c0ae200 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c826aa640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c13fb00 con 0x7f4c7c071260 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c826aa640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c150b30 con 0x7f4c7c15e5d0 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c80ea7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c0ae200 0x7f4c7c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c80ea7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c0ae200 0x7f4c7c09aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52870/0 (socket says 172.21.15.72:52870) 2023-12-12T09:53:40.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.479+0000 7f4c80ea7640 1 -- 172.21.15.72:0/1217284990 learned_addr learned my addr 172.21.15.72:0/1217284990 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c81ea9640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c7c15e5d0 0x7f4c7c09af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c816a8640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7c071260 0x7f4c7c13f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c80ea7640 1 -- 172.21.15.72:0/1217284990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c7c15e5d0 msgr2=0x7f4c7c09af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c80ea7640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c7c15e5d0 0x7f4c7c09af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c80ea7640 1 -- 172.21.15.72:0/1217284990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7c071260 msgr2=0x7f4c7c13f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c80ea7640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7c071260 0x7f4c7c13f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c80ea7640 1 -- 172.21.15.72:0/1217284990 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c78009400 con 0x7f4c7c0ae200 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c80ea7640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c0ae200 0x7f4c7c09aa40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00c430 tx=0x7f4c6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c71ffb640 1 -- 172.21.15.72:0/1217284990 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011790 con 0x7f4c7c0ae200 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c71ffb640 1 -- 172.21.15.72:0/1217284990 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c0118f0 con 0x7f4c7c0ae200 2023-12-12T09:53:40.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c71ffb640 1 -- 172.21.15.72:0/1217284990 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011ce0 con 0x7f4c7c0ae200 2023-12-12T09:53:40.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.480+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c150e10 con 0x7f4c7c0ae200 2023-12-12T09:53:40.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.481+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c14b0b0 con 0x7f4c7c0ae200 2023-12-12T09:53:40.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.482+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c1533b0 con 0x7f4c7c0ae200 2023-12-12T09:53:40.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.483+0000 7f4c71ffb640 1 -- 172.21.15.72:0/1217284990 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c6c01c460 con 0x7f4c7c0ae200 2023-12-12T09:53:40.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.484+0000 7f4c71ffb640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4c6006ae70 0x7f4c6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.484+0000 7f4c71ffb640 1 -- 172.21.15.72:0/1217284990 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4c6c089320 con 0x7f4c7c0ae200 2023-12-12T09:53:40.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.484+0000 7f4c816a8640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4c6006ae70 0x7f4c6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:40.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.485+0000 7f4c816a8640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4c6006ae70 0x7f4c6006d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4c78034000 tx=0x7f4c78008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:40.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.488+0000 7f4c71ffb640 1 -- 172.21.15.72:0/1217284990 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c6c0545c0 con 0x7f4c7c0ae200 2023-12-12T09:53:40.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.752+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f4c7c074450 con 0x7f4c7c0ae200 2023-12-12T09:53:40.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.757+0000 7f4c71ffb640 1 -- 172.21.15.72:0/1217284990 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4c6c087e60 con 0x7f4c7c0ae200 2023-12-12T09:53:40.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4c6006ae70 msgr2=0x7f4c6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4c6006ae70 0x7f4c6006d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4c78034000 tx=0x7f4c78008030 comp rx=0 tx=0).stop 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c0ae200 msgr2=0x7f4c7c09aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c0ae200 0x7f4c7c09aa40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00c430 tx=0x7f4c6c00c8f0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 shutdown_connections 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4c6006ae70 0x7f4c6006d330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c7c15e5d0 0x7f4c7c09af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c7c0ae200 0x7f4c7c09aa40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 --2- 172.21.15.72:0/1217284990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7c071260 0x7f4c7c13f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.759+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 >> 172.21.15.72:0/1217284990 conn(0x7f4c7c0b0860 msgr2=0x7f4c7c0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.760+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 shutdown_connections 2023-12-12T09:53:40.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.760+0000 7f4c826aa640 1 -- 172.21.15.72:0/1217284990 wait complete. 2023-12-12T09:53:40.780 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-12T09:53:40.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.983+0000 7f43d7fff640 1 Processor -- start 2023-12-12T09:53:40.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.983+0000 7f43d7fff640 1 -- start start 2023-12-12T09:53:40.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.985+0000 7f43d7fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f43d8070750 0x7f43d80ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.985+0000 7f43d7fff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d8070070 con 0x7f43d80713b0 2023-12-12T09:53:40.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.985+0000 7f43d7fff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d8070310 con 0x7f43d8159fd0 2023-12-12T09:53:40.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.985+0000 7f43d7fff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d80705f0 con 0x7f43d8070750 2023-12-12T09:53:40.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.986+0000 7f43d77fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f43d8070750 0x7f43d80ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:40.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.986+0000 7f43ce7fc640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f43d80713b0 0x7f43d80b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59392/0 (socket says 172.21.15.72:59392) 2023-12-12T09:53:40.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.986+0000 7f43ce7fc640 1 -- 172.21.15.72:0/1069516637 learned_addr learned my addr 172.21.15.72:0/1069516637 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:40.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.986+0000 7f43d67fc640 1 -- 172.21.15.72:0/1069516637 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088508426 0 0) 0x7f43d8070070 con 0x7f43d80713b0 2023-12-12T09:53:40.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.986+0000 7f43d77fe640 1 -- 172.21.15.72:0/1069516637 >> v1:172.21.15.113:6789/0 conn(0x7f43d8159fd0 legacy=0x7f43d8070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:40.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.986+0000 7f43d77fe640 1 -- 172.21.15.72:0/1069516637 >> v1:172.21.15.72:6789/0 conn(0x7f43d80713b0 legacy=0x7f43d80b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.986+0000 7f43d77fe640 1 -- 172.21.15.72:0/1069516637 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d80b1d00 con 0x7f43d8070750 2023-12-12T09:53:40.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.987+0000 7f43d77fe640 1 --2- 172.21.15.72:0/1069516637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d80ae1d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f43d00097a0 tx=0x7f43d002b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c12be1674b136ec server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:40.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d67fc640 1 -- 172.21.15.72:0/1069516637 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d002f810 con 0x7f43d8070750 2023-12-12T09:53:40.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d67fc640 1 -- 172.21.15.72:0/1069516637 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d002f970 con 0x7f43d8070750 2023-12-12T09:53:40.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d67fc640 1 -- 172.21.15.72:0/1069516637 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d002fd00 con 0x7f43d8070750 2023-12-12T09:53:40.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d7fff640 1 -- 172.21.15.72:0/1069516637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 msgr2=0x7f43d80ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d7fff640 1 --2- 172.21.15.72:0/1069516637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d80ae1d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f43d00097a0 tx=0x7f43d002b9f0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d7fff640 1 -- 172.21.15.72:0/1069516637 shutdown_connections 2023-12-12T09:53:40.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d7fff640 1 --2- 172.21.15.72:0/1069516637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d80ae1d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d7fff640 1 -- 172.21.15.72:0/1069516637 >> 172.21.15.72:0/1069516637 conn(0x7f43d80b0860 msgr2=0x7f43d80b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:40.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d7fff640 1 -- 172.21.15.72:0/1069516637 shutdown_connections 2023-12-12T09:53:40.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.988+0000 7f43d7fff640 1 -- 172.21.15.72:0/1069516637 wait complete. 2023-12-12T09:53:40.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.990+0000 7f43d7fff640 1 Processor -- start 2023-12-12T09:53:40.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.990+0000 7f43d7fff640 1 -- start start 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d814f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d80713b0 0x7f43d81f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43d8159fd0 0x7f43d809bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d7fff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80999f0 con 0x7f43d8070750 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d7fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d809a570 con 0x7f43d80713b0 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d7fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8099c40 con 0x7f43d8159fd0 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d6ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d814f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d6ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d814f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52886/0 (socket says 172.21.15.72:52886) 2023-12-12T09:53:40.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.991+0000 7f43d6ffd640 1 -- 172.21.15.72:0/2137410522 learned_addr learned my addr 172.21.15.72:0/2137410522 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43d6ffd640 1 -- 172.21.15.72:0/2137410522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43d8159fd0 msgr2=0x7f43d809bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43d6ffd640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43d8159fd0 0x7f43d809bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43d6ffd640 1 -- 172.21.15.72:0/2137410522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d80713b0 msgr2=0x7f43d81f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43d6ffd640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d80713b0 0x7f43d81f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43d6ffd640 1 -- 172.21.15.72:0/2137410522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0009400 con 0x7f43d8070750 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43d6ffd640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d814f100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c400 tx=0x7f43c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43cffff640 1 -- 172.21.15.72:0/2137410522 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c80052d0 con 0x7f43d8070750 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43cffff640 1 -- 172.21.15.72:0/2137410522 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8005430 con 0x7f43d8070750 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d8099f20 con 0x7f43d8070750 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43cffff640 1 -- 172.21.15.72:0/2137410522 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8003830 con 0x7f43d8070750 2023-12-12T09:53:40.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.992+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d81f2110 con 0x7f43d8070750 2023-12-12T09:53:40.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.993+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d81f2330 con 0x7f43d8070750 2023-12-12T09:53:40.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.994+0000 7f43cffff640 1 -- 172.21.15.72:0/2137410522 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43c8003990 con 0x7f43d8070750 2023-12-12T09:53:40.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.995+0000 7f43cffff640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43b806ae40 0x7f43b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:40.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.995+0000 7f43cffff640 1 -- 172.21.15.72:0/2137410522 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f43c8088a20 con 0x7f43d8070750 2023-12-12T09:53:40.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.995+0000 7f43ce7fc640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43b806ae40 0x7f43b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:40.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.996+0000 7f43ce7fc640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43b806ae40 0x7f43b806d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000f80 tx=0x7f43c0007c80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:41.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:40.999+0000 7f43cffff640 1 -- 172.21.15.72:0/2137410522 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43c8053c40 con 0x7f43d8070750 2023-12-12T09:53:41.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.248+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f43d8074450 con 0x7f43d8070750 2023-12-12T09:53:41.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.253+0000 7f43cffff640 1 -- 172.21.15.72:0/2137410522 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f43c80170a0 con 0x7f43d8070750 2023-12-12T09:53:41.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.255+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43b806ae40 msgr2=0x7f43b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.255+0000 7f43d7fff640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43b806ae40 0x7f43b806d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000f80 tx=0x7f43c0007c80 comp rx=0 tx=0).stop 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.255+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 msgr2=0x7f43d814f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.255+0000 7f43d7fff640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d814f100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c400 tx=0x7f43c800c8c0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.256+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 shutdown_connections 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.256+0000 7f43d7fff640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f43b806ae40 0x7f43b806d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.256+0000 7f43d7fff640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43d8159fd0 0x7f43d809bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.256+0000 7f43d7fff640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d80713b0 0x7f43d81f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.256+0000 7f43d7fff640 1 --2- 172.21.15.72:0/2137410522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43d8070750 0x7f43d814f100 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.256+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 >> 172.21.15.72:0/2137410522 conn(0x7f43d80b0860 msgr2=0x7f43d80ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.256+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 shutdown_connections 2023-12-12T09:53:41.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.256+0000 7f43d7fff640 1 -- 172.21.15.72:0/2137410522 wait complete. 2023-12-12T09:53:41.274 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-12T09:53:41.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.477+0000 7f444496e640 1 Processor -- start 2023-12-12T09:53:41.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.477+0000 7f444496e640 1 -- start start 2023-12-12T09:53:41.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f444496e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f444006c410 0x7f444015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f444496e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f444015e0e0 0x7f44400cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f444496e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444006ca40 con 0x7f4440157a60 2023-12-12T09:53:41.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f444496e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444015e550 con 0x7f444015e0e0 2023-12-12T09:53:41.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f444496e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4440071cd0 con 0x7f444006c410 2023-12-12T09:53:41.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f443e59c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4440157a60 0x7f444006de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59396/0 (socket says 172.21.15.72:59396) 2023-12-12T09:53:41.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f443e59c640 1 -- 172.21.15.72:0/1057241639 learned_addr learned my addr 172.21.15.72:0/1057241639 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170214234 0 0) 0x7f444006ca40 con 0x7f4440157a60 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f442c003520 con 0x7f4440157a60 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f443ed9d640 1 --2- 172.21.15.72:0/1057241639 >> v2:172.21.15.113:3300/0 conn(0x7f444015e0e0 0x7f44400cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.479+0000 7f4437fff640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f444006c410 0x7f444015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899655663 0 0) 0x7f442c003520 con 0x7f4440157a60 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f444006c410 msgr2=0x7f444015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f444006c410 0x7f444015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 >> v2:172.21.15.113:3300/0 conn(0x7f444015e0e0 msgr2=0x7f44400cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 --2- 172.21.15.72:0/1057241639 >> v2:172.21.15.113:3300/0 conn(0x7f444015e0e0 0x7f44400cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440070f10 con 0x7f4440157a60 2023-12-12T09:53:41.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f4430001670 con 0x7f4440157a60 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 >> v1:172.21.15.72:6789/0 conn(0x7f4440157a60 legacy=0x7f444006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f442c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c0047c0 0x7f442c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f444006c410 0x7f442c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444015e550 con 0x7f442c003970 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4440071cd0 con 0x7f442c0047c0 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442c00ddd0 con 0x7f444006c410 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443e59c640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f442c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443e59c640 1 -- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f444006c410 msgr2=0x7f442c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443e59c640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f444006c410 0x7f442c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443e59c640 1 -- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c0047c0 msgr2=0x7f442c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:41.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443e59c640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c0047c0 0x7f442c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.480+0000 7f443e59c640 1 -- 172.21.15.72:0/1057241639 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c003520 con 0x7f442c003970 2023-12-12T09:53:41.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.481+0000 7f443e59c640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f442c003d90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f443000c110 tx=0x7f442c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6040879dd80e7700 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:41.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430005ad0 con 0x7f442c003970 2023-12-12T09:53:41.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4430005c50 con 0x7f442c003970 2023-12-12T09:53:41.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f443dd9b640 1 -- 172.21.15.72:0/1057241639 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443002c830 con 0x7f442c003970 2023-12-12T09:53:41.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f444496e640 1 -- 172.21.15.72:0/1057241639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 msgr2=0x7f442c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f444496e640 1 --2- 172.21.15.72:0/1057241639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f442c003d90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f443000c110 tx=0x7f442c0057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f443ed9d640 1 -- 172.21.15.72:0/1057241639 reap_dead start 2023-12-12T09:53:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f444496e640 1 -- 172.21.15.72:0/1057241639 shutdown_connections 2023-12-12T09:53:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f444496e640 1 -- 172.21.15.72:0/1057241639 >> 172.21.15.72:0/1057241639 conn(0x7f44400b0a90 msgr2=0x7f44400b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f444496e640 1 -- 172.21.15.72:0/1057241639 shutdown_connections 2023-12-12T09:53:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.482+0000 7f444496e640 1 -- 172.21.15.72:0/1057241639 wait complete. 2023-12-12T09:53:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.484+0000 7f444496e640 1 Processor -- start 2023-12-12T09:53:41.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.484+0000 7f444496e640 1 -- start start 2023-12-12T09:53:41.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f444496e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f444013beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f444496e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4440155930 0x7f4440138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f444496e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444006cca0 0x7f444006d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f444496e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444013c4d0 con 0x7f442c003970 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f444496e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444006d660 con 0x7f444006cca0 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f444496e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444015be90 con 0x7f4440155930 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f443e59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f444013beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f443e59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f444013beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52906/0 (socket says 172.21.15.72:52906) 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.485+0000 7f443e59c640 1 -- 172.21.15.72:0/2164199761 learned_addr learned my addr 172.21.15.72:0/2164199761 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f443e59c640 1 -- 172.21.15.72:0/2164199761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4440155930 msgr2=0x7f4440138ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f443e59c640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4440155930 0x7f4440138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f443e59c640 1 -- 172.21.15.72:0/2164199761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444006cca0 msgr2=0x7f444006d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f443e59c640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444006cca0 0x7f444006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f443e59c640 1 -- 172.21.15.72:0/2164199761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444014adb0 con 0x7f442c003970 2023-12-12T09:53:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f443e59c640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f444013beb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f443002c3d0 tx=0x7f4430001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f4436ffd640 1 -- 172.21.15.72:0/2164199761 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443003c040 con 0x7f442c003970 2023-12-12T09:53:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f4436ffd640 1 -- 172.21.15.72:0/2164199761 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443002dd50 con 0x7f442c003970 2023-12-12T09:53:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4440155d90 con 0x7f442c003970 2023-12-12T09:53:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f4436ffd640 1 -- 172.21.15.72:0/2164199761 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44300072b0 con 0x7f442c003970 2023-12-12T09:53:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.486+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444015d530 con 0x7f442c003970 2023-12-12T09:53:41.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.487+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44400745d0 con 0x7f442c003970 2023-12-12T09:53:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.488+0000 7f4436ffd640 1 -- 172.21.15.72:0/2164199761 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4430032070 con 0x7f442c003970 2023-12-12T09:53:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.489+0000 7f4436ffd640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f442006ae70 0x7f442006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.489+0000 7f4436ffd640 1 -- 172.21.15.72:0/2164199761 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f44300aad50 con 0x7f442c003970 2023-12-12T09:53:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.489+0000 7f4437fff640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f442006ae70 0x7f442006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:41.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.490+0000 7f4437fff640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f442006ae70 0x7f442006d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4428006f00 tx=0x7f4428008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:41.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.493+0000 7f4436ffd640 1 -- 172.21.15.72:0/2164199761 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4430075f60 con 0x7f442c003970 2023-12-12T09:53:41.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.735+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f4440155d50 con 0x7f442c003970 2023-12-12T09:53:41.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.741+0000 7f4436ffd640 1 -- 172.21.15.72:0/2164199761 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4430079580 con 0x7f442c003970 2023-12-12T09:53:41.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.743+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f442006ae70 msgr2=0x7f442006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.743+0000 7f444496e640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f442006ae70 0x7f442006d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4428006f00 tx=0x7f4428008040 comp rx=0 tx=0).stop 2023-12-12T09:53:41.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.743+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 msgr2=0x7f444013beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.743+0000 7f444496e640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f444013beb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f443002c3d0 tx=0x7f4430001930 comp rx=0 tx=0).stop 2023-12-12T09:53:41.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.744+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 shutdown_connections 2023-12-12T09:53:41.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.744+0000 7f444496e640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f442006ae70 0x7f442006d330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.744+0000 7f444496e640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444006cca0 0x7f444006d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.744+0000 7f444496e640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4440155930 0x7f4440138ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.744+0000 7f444496e640 1 --2- 172.21.15.72:0/2164199761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f442c003970 0x7f444013beb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.744+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 >> 172.21.15.72:0/2164199761 conn(0x7f44400b0a90 msgr2=0x7f4440154d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:41.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.744+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 shutdown_connections 2023-12-12T09:53:41.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.744+0000 7f444496e640 1 -- 172.21.15.72:0/2164199761 wait complete. 2023-12-12T09:53:41.764 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-12T09:53:41.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.983+0000 7fa7bbf41640 1 Processor -- start 2023-12-12T09:53:41.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.983+0000 7fa7bbf41640 1 -- start start 2023-12-12T09:53:41.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.985+0000 7fa7bbf41640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7b40cd000 0x7fa7b406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.985+0000 7fa7bbf41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 0x7fa7b406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:41.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.985+0000 7fa7bbf41640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b406d260 con 0x7fa7b41467c0 2023-12-12T09:53:41.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.985+0000 7fa7bbf41640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b40ac860 con 0x7fa7b40cd000 2023-12-12T09:53:41.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.985+0000 7fa7bbf41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b406d510 con 0x7fa7b4146ba0 2023-12-12T09:53:41.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.986+0000 7fa7b94dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7b40cd000 0x7fa7b406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:41.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.986+0000 7fa7ba4de640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 0x7fa7b406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:41.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.986+0000 7fa7b94dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7b40cd000 0x7fa7b406c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:34766/0 (socket says 172.21.15.72:34766) 2023-12-12T09:53:41.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.986+0000 7fa7b94dc640 1 -- 172.21.15.72:0/3178550576 learned_addr learned my addr 172.21.15.72:0/3178550576 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:41.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.986+0000 7fa7b94dc640 1 -- 172.21.15.72:0/3178550576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 msgr2=0x7fa7b406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:41.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.986+0000 7fa7b94dc640 1 --2- 172.21.15.72:0/3178550576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 0x7fa7b406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.986+0000 7fa7b94dc640 1 -- 172.21.15.72:0/3178550576 >> v1:172.21.15.113:6789/0 conn(0x7fa7b41467c0 legacy=0x7fa7b40cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.986+0000 7fa7b94dc640 1 -- 172.21.15.72:0/3178550576 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b406d7f0 con 0x7fa7b40cd000 2023-12-12T09:53:41.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.988+0000 7fa7b94dc640 1 --2- 172.21.15.72:0/3178550576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b40cd000 0x7fa7b406c7e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009870 tx=0x7fa7a402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd8dc529d6e3525c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.988+0000 7fa7b8cdb640 1 -- 172.21.15.72:0/3178550576 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4030840 con 0x7fa7b40cd000 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.988+0000 7fa7b8cdb640 1 -- 172.21.15.72:0/3178550576 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a40309a0 con 0x7fa7b40cd000 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.988+0000 7fa7b8cdb640 1 -- 172.21.15.72:0/3178550576 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4030d30 con 0x7fa7b40cd000 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.989+0000 7fa7bbf41640 1 -- 172.21.15.72:0/3178550576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b40cd000 msgr2=0x7fa7b406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.989+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/3178550576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b40cd000 0x7fa7b406c7e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009870 tx=0x7fa7a402b980 comp rx=0 tx=0).stop 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.989+0000 7fa7bbf41640 1 -- 172.21.15.72:0/3178550576 shutdown_connections 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.989+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/3178550576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 0x7fa7b406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.989+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/3178550576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b40cd000 0x7fa7b406c7e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.989+0000 7fa7bbf41640 1 -- 172.21.15.72:0/3178550576 >> 172.21.15.72:0/3178550576 conn(0x7fa7b40b0a90 msgr2=0x7fa7b40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.989+0000 7fa7bbf41640 1 -- 172.21.15.72:0/3178550576 shutdown_connections 2023-12-12T09:53:41.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.989+0000 7fa7bbf41640 1 -- 172.21.15.72:0/3178550576 wait complete. 2023-12-12T09:53:41.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.991+0000 7fa7bbf41640 1 Processor -- start 2023-12-12T09:53:41.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.991+0000 7fa7bbf41640 1 -- start start 2023-12-12T09:53:42.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7bbf41640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7b40cd000 0x7fa7b4156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7bbf41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b41467c0 0x7fa7b41e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7bbf41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 0x7fa7b41e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7bbf41640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40b23a0 con 0x7fa7b40cd000 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7bbf41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b41585a0 con 0x7fa7b41467c0 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7bbf41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b41582c0 con 0x7fa7b4146ba0 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7b9cdd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7b40cd000 0x7fa7b4156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7b9cdd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7b40cd000 0x7fa7b4156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52916/0 (socket says 172.21.15.72:52916) 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.992+0000 7fa7b9cdd640 1 -- 172.21.15.72:0/1333605738 learned_addr learned my addr 172.21.15.72:0/1333605738 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7b94dc640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b41467c0 0x7fa7b41e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7b9cdd640 1 -- 172.21.15.72:0/1333605738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 msgr2=0x7fa7b41e81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7b9cdd640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 0x7fa7b41e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7b9cdd640 1 -- 172.21.15.72:0/1333605738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b41467c0 msgr2=0x7fa7b41e7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7b9cdd640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b41467c0 0x7fa7b41e7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7b9cdd640 1 -- 172.21.15.72:0/1333605738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a40094d0 con 0x7fa7b40cd000 2023-12-12T09:53:42.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7b94dc640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b41467c0 0x7fa7b41e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:42.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7b9cdd640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7b40cd000 0x7fa7b4156c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000c450 tx=0x7fa7b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:42.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7aa7fc640 1 -- 172.21.15.72:0/1333605738 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00052d0 con 0x7fa7b40cd000 2023-12-12T09:53:42.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7aa7fc640 1 -- 172.21.15.72:0/1333605738 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b0005430 con 0x7fa7b40cd000 2023-12-12T09:53:42.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b4132960 con 0x7fa7b40cd000 2023-12-12T09:53:42.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7aa7fc640 1 -- 172.21.15.72:0/1333605738 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0003830 con 0x7fa7b40cd000 2023-12-12T09:53:42.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.993+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b4132f20 con 0x7fa7b40cd000 2023-12-12T09:53:42.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.995+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b41327c0 con 0x7fa7b40cd000 2023-12-12T09:53:42.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.995+0000 7fa7aa7fc640 1 -- 172.21.15.72:0/1333605738 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7b0003990 con 0x7fa7b40cd000 2023-12-12T09:53:42.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.996+0000 7fa7aa7fc640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa79806ae40 0x7fa79806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.996+0000 7fa7aa7fc640 1 -- 172.21.15.72:0/1333605738 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa7b0088a50 con 0x7fa7b40cd000 2023-12-12T09:53:42.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.996+0000 7fa7b94dc640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa79806ae40 0x7fa79806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:41.997+0000 7fa7b94dc640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa79806ae40 0x7fa79806d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4036000 tx=0x7fa7a403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:42.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.000+0000 7fa7aa7fc640 1 -- 172.21.15.72:0/1333605738 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7b41327c0 con 0x7fa7b40cd000 2023-12-12T09:53:42.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.237+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa7b41327c0 con 0x7fa7b40cd000 2023-12-12T09:53:42.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.243+0000 7fa7aa7fc640 1 -- 172.21.15.72:0/1333605738 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa7b41327c0 con 0x7fa7b40cd000 2023-12-12T09:53:42.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa79806ae40 msgr2=0x7fa79806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa79806ae40 0x7fa79806d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4036000 tx=0x7fa7a403b040 comp rx=0 tx=0).stop 2023-12-12T09:53:42.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7b40cd000 msgr2=0x7fa7b4156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7b40cd000 0x7fa7b4156c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000c450 tx=0x7fa7b000c910 comp rx=0 tx=0).stop 2023-12-12T09:53:42.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 shutdown_connections 2023-12-12T09:53:42.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa79806ae40 0x7fa79806d300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7b4146ba0 0x7fa7b41e81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7b41467c0 0x7fa7b41e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 --2- 172.21.15.72:0/1333605738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7b40cd000 0x7fa7b4156c10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 >> 172.21.15.72:0/1333605738 conn(0x7fa7b40b0a90 msgr2=0x7fa7b40cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:42.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 shutdown_connections 2023-12-12T09:53:42.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.246+0000 7fa7bbf41640 1 -- 172.21.15.72:0/1333605738 wait complete. 2023-12-12T09:53:42.267 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-12T09:53:42.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.472+0000 7f8bea1be640 1 Processor -- start 2023-12-12T09:53:42.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.472+0000 7f8bea1be640 1 -- start start 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.473+0000 7f8bea1be640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8be4157a60 0x7f8be406c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.473+0000 7f8bea1be640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 0x7f8be415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.473+0000 7f8bea1be640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8be415e0e0 0x7f8be415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.473+0000 7f8bea1be640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be415e550 con 0x7f8be4157a60 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.473+0000 7f8bea1be640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be406ca40 con 0x7f8be415e0e0 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.473+0000 7f8bea1be640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be4071cd0 con 0x7f8be406de20 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be37fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8be4157a60 0x7f8be406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be37fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8be4157a60 0x7f8be406c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52920/0 (socket says 172.21.15.72:52920) 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be37fe640 1 -- 172.21.15.72:0/3297057397 learned_addr learned my addr 172.21.15.72:0/3297057397 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be37fe640 1 -- 172.21.15.72:0/3297057397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 msgr2=0x7f8be415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:42.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be37fe640 1 --2- 172.21.15.72:0/3297057397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 0x7f8be415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be37fe640 1 -- 172.21.15.72:0/3297057397 >> v2:172.21.15.113:3300/0 conn(0x7f8be415e0e0 msgr2=0x7f8be415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:42.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be3fff640 1 --2- 172.21.15.72:0/3297057397 >> v2:172.21.15.113:3300/0 conn(0x7f8be415e0e0 0x7f8be415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be37fe640 1 --2- 172.21.15.72:0/3297057397 >> v2:172.21.15.113:3300/0 conn(0x7f8be415e0e0 0x7f8be415a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.474+0000 7f8be37fe640 1 -- 172.21.15.72:0/3297057397 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4070f10 con 0x7f8be4157a60 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.475+0000 7f8be37fe640 1 --2- 172.21.15.72:0/3297057397 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be406c410 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8be4070150 tx=0x7f8bd402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7711520e4800e612 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.475+0000 7f8be27fc640 1 -- 172.21.15.72:0/3297057397 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd402c9f0 con 0x7f8be4157a60 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8be27fc640 1 -- 172.21.15.72:0/3297057397 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd402cb50 con 0x7f8be4157a60 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8be27fc640 1 -- 172.21.15.72:0/3297057397 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40343f0 con 0x7f8be4157a60 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 -- 172.21.15.72:0/3297057397 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 msgr2=0x7f8be406c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3297057397 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be406c410 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8be4070150 tx=0x7f8bd402be50 comp rx=0 tx=0).stop 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 -- 172.21.15.72:0/3297057397 shutdown_connections 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3297057397 >> v2:172.21.15.113:3300/0 conn(0x7f8be415e0e0 0x7f8be415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3297057397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 0x7f8be415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3297057397 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be406c410 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 -- 172.21.15.72:0/3297057397 >> 172.21.15.72:0/3297057397 conn(0x7f8be40b0a90 msgr2=0x7f8be40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:42.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 -- 172.21.15.72:0/3297057397 shutdown_connections 2023-12-12T09:53:42.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.476+0000 7f8bea1be640 1 -- 172.21.15.72:0/3297057397 wait complete. 2023-12-12T09:53:42.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.478+0000 7f8bea1be640 1 Processor -- start 2023-12-12T09:53:42.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.478+0000 7f8bea1be640 1 -- start start 2023-12-12T09:53:42.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8bea1be640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 0x7f8be414e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8bea1be640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be40b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8bea1be640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be415e0e0 0x7f8be40c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8bea1be640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be414ecf0 con 0x7f8be4157a60 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8bea1be640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40b34f0 con 0x7f8be415e0e0 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8bea1be640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40b37d0 con 0x7f8be406de20 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be40b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be40b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52932/0 (socket says 172.21.15.72:52932) 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.479+0000 7f8be2ffd640 1 -- 172.21.15.72:0/3625894527 learned_addr learned my addr 172.21.15.72:0/3625894527 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8be37fe640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 0x7f8be414e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8be2ffd640 1 -- 172.21.15.72:0/3625894527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 msgr2=0x7f8be414e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8be2ffd640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 0x7f8be414e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8be2ffd640 1 -- 172.21.15.72:0/3625894527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be415e0e0 msgr2=0x7f8be40c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8be3fff640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be415e0e0 0x7f8be40c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8be2ffd640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be415e0e0 0x7f8be40c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8be2ffd640 1 -- 172.21.15.72:0/3625894527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd4009b30 con 0x7f8be4157a60 2023-12-12T09:53:42.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8be2ffd640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be40b2c10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00b4a0 tx=0x7f8bcc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:42.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8bdbfff640 1 -- 172.21.15.72:0/3625894527 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc010610 con 0x7f8be4157a60 2023-12-12T09:53:42.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8bdbfff640 1 -- 172.21.15.72:0/3625894527 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc010770 con 0x7f8be4157a60 2023-12-12T09:53:42.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8bdbfff640 1 -- 172.21.15.72:0/3625894527 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc010b20 con 0x7f8be4157a60 2023-12-12T09:53:42.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be40b3ab0 con 0x7f8be4157a60 2023-12-12T09:53:42.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.480+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be40b4070 con 0x7f8be4157a60 2023-12-12T09:53:42.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.482+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be40745d0 con 0x7f8be4157a60 2023-12-12T09:53:42.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.483+0000 7f8bdbfff640 1 -- 172.21.15.72:0/3625894527 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8bcc010c80 con 0x7f8be4157a60 2023-12-12T09:53:42.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.483+0000 7f8bdbfff640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8bc406ae40 0x7f8bc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.484+0000 7f8bdbfff640 1 -- 172.21.15.72:0/3625894527 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8bcc0890c0 con 0x7f8be4157a60 2023-12-12T09:53:42.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.484+0000 7f8be37fe640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8bc406ae40 0x7f8bc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.485+0000 7f8be37fe640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8bc406ae40 0x7f8bc406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4035000 tx=0x7f8bd4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:42.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.488+0000 7f8bdbfff640 1 -- 172.21.15.72:0/3625894527 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8bcc0542e0 con 0x7f8be4157a60 2023-12-12T09:53:42.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.744+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8be40cc890 con 0x7f8be4157a60 2023-12-12T09:53:42.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.750+0000 7f8bdbfff640 1 -- 172.21.15.72:0/3625894527 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8bcc057900 con 0x7f8be4157a60 2023-12-12T09:53:42.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8bc406ae40 msgr2=0x7f8bc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8bc406ae40 0x7f8bc406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4035000 tx=0x7f8bd4030000 comp rx=0 tx=0).stop 2023-12-12T09:53:42.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 msgr2=0x7f8be40b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be40b2c10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00b4a0 tx=0x7f8bcc00b960 comp rx=0 tx=0).stop 2023-12-12T09:53:42.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 shutdown_connections 2023-12-12T09:53:42.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8bc406ae40 0x7f8bc406d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be415e0e0 0x7f8be40c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8be4157a60 0x7f8be40b2c10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 --2- 172.21.15.72:0/3625894527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be406de20 0x7f8be414e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 >> 172.21.15.72:0/3625894527 conn(0x7f8be40b0a90 msgr2=0x7f8be40cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.752+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 shutdown_connections 2023-12-12T09:53:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.753+0000 7f8bea1be640 1 -- 172.21.15.72:0/3625894527 wait complete. 2023-12-12T09:53:42.773 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-12T09:53:42.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.975+0000 7fdcf4d33640 1 Processor -- start 2023-12-12T09:53:42.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.975+0000 7fdcf4d33640 1 -- start start 2023-12-12T09:53:42.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.977+0000 7fdcf4d33640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00cdeb0 0x7fdcf00cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.977+0000 7fdcf4d33640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdcf00ccb80 0x7fdcf00adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.977+0000 7fdcf4d33640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf00cc6f0 con 0x7fdcf015b3a0 2023-12-12T09:53:42.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.977+0000 7fdcf4d33640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf00cff80 con 0x7fdcf00ccb80 2023-12-12T09:53:42.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.977+0000 7fdcf4d33640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00cbdc0 con 0x7fdcf00cdeb0 2023-12-12T09:53:42.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.977+0000 7fdceed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdcf00ccb80 0x7fdcf00adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.977+0000 7fdceed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdcf00ccb80 0x7fdcf00adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52936/0 (socket says 172.21.15.72:52936) 2023-12-12T09:53:42.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.977+0000 7fdceed9d640 1 -- 172.21.15.72:0/876423979 learned_addr learned my addr 172.21.15.72:0/876423979 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:42.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.978+0000 7fdce7fff640 1 --2- 172.21.15.72:0/876423979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00cdeb0 0x7fdcf00cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.978+0000 7fdceed9d640 1 -- 172.21.15.72:0/876423979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00cdeb0 msgr2=0x7fdcf00cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:42.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.978+0000 7fdceed9d640 1 --2- 172.21.15.72:0/876423979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00cdeb0 0x7fdcf00cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.978+0000 7fdceed9d640 1 -- 172.21.15.72:0/876423979 >> v1:172.21.15.113:6789/0 conn(0x7fdcf015b3a0 legacy=0x7fdcf00cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:42.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.978+0000 7fdceed9d640 1 -- 172.21.15.72:0/876423979 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf00cc0a0 con 0x7fdcf00ccb80 2023-12-12T09:53:42.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.978+0000 7fdce7fff640 1 --2- 172.21.15.72:0/876423979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00cdeb0 0x7fdcf00cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.979+0000 7fdceed9d640 1 --2- 172.21.15.72:0/876423979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf00adc60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009e50 tx=0x7fdce802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a8d17079f9d869 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.979+0000 7fdcedd9b640 1 -- 172.21.15.72:0/876423979 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce802ca80 con 0x7fdcf00ccb80 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.979+0000 7fdcedd9b640 1 -- 172.21.15.72:0/876423979 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce802cbe0 con 0x7fdcf00ccb80 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.979+0000 7fdcedd9b640 1 -- 172.21.15.72:0/876423979 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce80343f0 con 0x7fdcf00ccb80 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.980+0000 7fdcf4d33640 1 -- 172.21.15.72:0/876423979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 msgr2=0x7fdcf00adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.980+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/876423979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf00adc60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009e50 tx=0x7fdce802bee0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.980+0000 7fdcf4d33640 1 -- 172.21.15.72:0/876423979 shutdown_connections 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.980+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/876423979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf00adc60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.980+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/876423979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf00cdeb0 0x7fdcf00cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.980+0000 7fdcf4d33640 1 -- 172.21.15.72:0/876423979 >> 172.21.15.72:0/876423979 conn(0x7fdcf00d0ed0 msgr2=0x7fdcf00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.980+0000 7fdcf4d33640 1 -- 172.21.15.72:0/876423979 shutdown_connections 2023-12-12T09:53:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.980+0000 7fdcf4d33640 1 -- 172.21.15.72:0/876423979 wait complete. 2023-12-12T09:53:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.982+0000 7fdcf4d33640 1 Processor -- start 2023-12-12T09:53:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.982+0000 7fdcf4d33640 1 -- start start 2023-12-12T09:53:42.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcf4d33640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf0158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcf4d33640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00cdeb0 0x7fdcf00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcf4d33640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf015b3a0 0x7fdcf00b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcf4d33640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf006d580 con 0x7fdcf00ccb80 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcf4d33640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01f41f0 con 0x7fdcf00cdeb0 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcf4d33640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01f3430 con 0x7fdcf015b3a0 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcee59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf0158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcee59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf0158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52948/0 (socket says 172.21.15.72:52948) 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.983+0000 7fdcee59c640 1 -- 172.21.15.72:0/1069696189 learned_addr learned my addr 172.21.15.72:0/1069696189 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:42.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdceed9d640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf015b3a0 0x7fdcf00b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdcee59c640 1 -- 172.21.15.72:0/1069696189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf015b3a0 msgr2=0x7fdcf00b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdcee59c640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf015b3a0 0x7fdcf00b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdce7fff640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00cdeb0 0x7fdcf00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdcee59c640 1 -- 172.21.15.72:0/1069696189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00cdeb0 msgr2=0x7fdcf00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdcee59c640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00cdeb0 0x7fdcf00b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdcee59c640 1 -- 172.21.15.72:0/1069696189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8009b30 con 0x7fdcf00ccb80 2023-12-12T09:53:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdcee59c640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf0158800 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdce000b3f0 tx=0x7fdce000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdce6ffd640 1 -- 172.21.15.72:0/1069696189 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0011830 con 0x7fdcf00ccb80 2023-12-12T09:53:42.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdce6ffd640 1 -- 172.21.15.72:0/1069696189 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce0011990 con 0x7fdcf00ccb80 2023-12-12T09:53:42.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf01f3710 con 0x7fdcf00ccb80 2023-12-12T09:53:42.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdce6ffd640 1 -- 172.21.15.72:0/1069696189 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0011d40 con 0x7fdcf00ccb80 2023-12-12T09:53:42.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.984+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf01f3cd0 con 0x7fdcf00ccb80 2023-12-12T09:53:42.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.985+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf01f3ef0 con 0x7fdcf00ccb80 2023-12-12T09:53:42.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.986+0000 7fdce6ffd640 1 -- 172.21.15.72:0/1069696189 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdce001a400 con 0x7fdcf00ccb80 2023-12-12T09:53:42.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.988+0000 7fdce6ffd640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcd006ae70 0x7fdcd006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:42.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.988+0000 7fdce6ffd640 1 -- 172.21.15.72:0/1069696189 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdce00892a0 con 0x7fdcf00ccb80 2023-12-12T09:53:42.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.988+0000 7fdce7fff640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcd006ae70 0x7fdcd006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:42.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.988+0000 7fdce7fff640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcd006ae70 0x7fdcd006d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00b0200 tx=0x7fdcd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:42.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:42.992+0000 7fdce6ffd640 1 -- 172.21.15.72:0/1069696189 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdce00544c0 con 0x7fdcf00ccb80 2023-12-12T09:53:43.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.249+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fdcf00745d0 con 0x7fdcf00ccb80 2023-12-12T09:53:43.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.255+0000 7fdce6ffd640 1 -- 172.21.15.72:0/1069696189 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdce0057ae0 con 0x7fdcf00ccb80 2023-12-12T09:53:43.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcd006ae70 msgr2=0x7fdcd006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcd006ae70 0x7fdcd006d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00b0200 tx=0x7fdcd8008040 comp rx=0 tx=0).stop 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 msgr2=0x7fdcf0158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf0158800 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdce000b3f0 tx=0x7fdce000b8b0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 shutdown_connections 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcd006ae70 0x7fdcd006d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcf015b3a0 0x7fdcf00b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf00cdeb0 0x7fdcf00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 --2- 172.21.15.72:0/1069696189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf00ccb80 0x7fdcf0158800 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.257+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 >> 172.21.15.72:0/1069696189 conn(0x7fdcf00d0ed0 msgr2=0x7fdcf00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:43.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.258+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 shutdown_connections 2023-12-12T09:53:43.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.258+0000 7fdcf4d33640 1 -- 172.21.15.72:0/1069696189 wait complete. 2023-12-12T09:53:43.279 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:43.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.494+0000 7fb4ccb0f640 1 Processor -- start 2023-12-12T09:53:43.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.495+0000 7fb4ccb0f640 1 -- start start 2023-12-12T09:53:43.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4ccb0f640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb4c806d1c0 0x7fb4c80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4ccb0f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c80af320 0x7fb4c815f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4ccb0f640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c80cabd0 con 0x7fb4c806a460 2023-12-12T09:53:43.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4ccb0f640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c80caeb0 con 0x7fb4c806d1c0 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4ccb0f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80cb190 con 0x7fb4c80af320 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4bffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb4c806d1c0 0x7fb4c80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4bffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb4c806d1c0 0x7fb4c80aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52952/0 (socket says 172.21.15.72:52952) 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4bffff640 1 -- 172.21.15.72:0/2831416870 learned_addr learned my addr 172.21.15.72:0/2831416870 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.496+0000 7fb4c659c640 1 --1- 172.21.15.72:0/2831416870 >> v1:172.21.15.72:6789/0 conn(0x7fb4c806a460 0x7fb4c806a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59408/0 (socket says 172.21.15.72:59408) 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.497+0000 7fb4bffff640 1 -- 172.21.15.72:0/2831416870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c80af320 msgr2=0x7fb4c815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.497+0000 7fb4bffff640 1 --2- 172.21.15.72:0/2831416870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c80af320 0x7fb4c815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.497+0000 7fb4bffff640 1 -- 172.21.15.72:0/2831416870 >> v1:172.21.15.72:6789/0 conn(0x7fb4c806a460 legacy=0x7fb4c806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.497+0000 7fb4c5d9b640 1 -- 172.21.15.72:0/2831416870 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465451392 0 0) 0x7fb4c80cabd0 con 0x7fb4c806a460 2023-12-12T09:53:43.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.497+0000 7fb4bffff640 1 -- 172.21.15.72:0/2831416870 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c80cb470 con 0x7fb4c806d1c0 2023-12-12T09:53:43.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.498+0000 7fb4bffff640 1 --2- 172.21.15.72:0/2831416870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806d1c0 0x7fb4c80aede0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009870 tx=0x7fb4b002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61699d57b40cedf5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.498+0000 7fb4c5d9b640 1 -- 172.21.15.72:0/2831416870 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0030840 con 0x7fb4c806d1c0 2023-12-12T09:53:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.498+0000 7fb4c5d9b640 1 -- 172.21.15.72:0/2831416870 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b00309a0 con 0x7fb4c806d1c0 2023-12-12T09:53:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.498+0000 7fb4c5d9b640 1 -- 172.21.15.72:0/2831416870 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0030d30 con 0x7fb4c806d1c0 2023-12-12T09:53:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.499+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/2831416870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806d1c0 msgr2=0x7fb4c80aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.499+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/2831416870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806d1c0 0x7fb4c80aede0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009870 tx=0x7fb4b002b980 comp rx=0 tx=0).stop 2023-12-12T09:53:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.499+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/2831416870 shutdown_connections 2023-12-12T09:53:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.499+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/2831416870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c80af320 0x7fb4c815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.499+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/2831416870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806d1c0 0x7fb4c80aede0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.499+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/2831416870 >> 172.21.15.72:0/2831416870 conn(0x7fb4c80cc7d0 msgr2=0x7fb4c80ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.499+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/2831416870 shutdown_connections 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.499+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/2831416870 wait complete. 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.501+0000 7fb4ccb0f640 1 Processor -- start 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.501+0000 7fb4ccb0f640 1 -- start start 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4ccb0f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806a460 0x7fb4c814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4ccb0f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c806d1c0 0x7fb4c81f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4ccb0f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c80af320 0x7fb4c80b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4ccb0f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c8147c40 con 0x7fb4c806a460 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4ccb0f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c814e460 con 0x7fb4c80af320 2023-12-12T09:53:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4ccb0f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80c7150 con 0x7fb4c806d1c0 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4c659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806a460 0x7fb4c814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4c659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806a460 0x7fb4c814df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52956/0 (socket says 172.21.15.72:52956) 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4c659c640 1 -- 172.21.15.72:0/76375811 learned_addr learned my addr 172.21.15.72:0/76375811 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4c659c640 1 -- 172.21.15.72:0/76375811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c806d1c0 msgr2=0x7fb4c81f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4c659c640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c806d1c0 0x7fb4c81f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4c659c640 1 -- 172.21.15.72:0/76375811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c80af320 msgr2=0x7fb4c80b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4c659c640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c80af320 0x7fb4c80b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.502+0000 7fb4c659c640 1 -- 172.21.15.72:0/76375811 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00094d0 con 0x7fb4c806a460 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.503+0000 7fb4c659c640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806a460 0x7fb4c814df20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800c530 tx=0x7fb4b800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.503+0000 7fb4beffd640 1 -- 172.21.15.72:0/76375811 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80074e0 con 0x7fb4c806a460 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.503+0000 7fb4beffd640 1 -- 172.21.15.72:0/76375811 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b8007640 con 0x7fb4c806a460 2023-12-12T09:53:43.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.503+0000 7fb4beffd640 1 -- 172.21.15.72:0/76375811 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8004a20 con 0x7fb4c806a460 2023-12-12T09:53:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.503+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c81f3190 con 0x7fb4c806a460 2023-12-12T09:53:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.503+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c81f75e0 con 0x7fb4c806a460 2023-12-12T09:53:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.504+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c80745d0 con 0x7fb4c806a460 2023-12-12T09:53:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.505+0000 7fb4beffd640 1 -- 172.21.15.72:0/76375811 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb4b8004b80 con 0x7fb4c806a460 2023-12-12T09:53:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.506+0000 7fb4beffd640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb4a806ae40 0x7fb4a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.506+0000 7fb4beffd640 1 -- 172.21.15.72:0/76375811 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb4b80885b0 con 0x7fb4c806a460 2023-12-12T09:53:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.506+0000 7fb4bffff640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb4a806ae40 0x7fb4a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:43.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.507+0000 7fb4bffff640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb4a806ae40 0x7fb4a806d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0036000 tx=0x7fb4b003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:43.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.510+0000 7fb4beffd640 1 -- 172.21.15.72:0/76375811 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4b8053860 con 0x7fb4c806a460 2023-12-12T09:53:43.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.707+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4c80cd4d0 con 0x7fb4c806a460 2023-12-12T09:53:43.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.708+0000 7fb4beffd640 1 -- 172.21.15.72:0/76375811 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13062 (secure 0 0 0) 0x7fb4b8018090 con 0x7fb4c806a460 2023-12-12T09:53:43.711 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:43.712 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":34,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:53:39.747598+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:43.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb4a806ae40 msgr2=0x7fb4a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb4a806ae40 0x7fb4a806d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0036000 tx=0x7fb4b003b040 comp rx=0 tx=0).stop 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806a460 msgr2=0x7fb4c814df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806a460 0x7fb4c814df20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800c530 tx=0x7fb4b800c9f0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 shutdown_connections 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb4a806ae40 0x7fb4a806d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c80af320 0x7fb4c80b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c806d1c0 0x7fb4c81f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 --2- 172.21.15.72:0/76375811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c806a460 0x7fb4c814df20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.711+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 >> 172.21.15.72:0/76375811 conn(0x7fb4c80cc7d0 msgr2=0x7fb4c80ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.712+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 shutdown_connections 2023-12-12T09:53:43.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.712+0000 7fb4ccb0f640 1 -- 172.21.15.72:0/76375811 wait complete. 2023-12-12T09:53:43.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:53:43.733 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:53:43.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.951+0000 7f7bf8389640 1 Processor -- start 2023-12-12T09:53:43.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.951+0000 7f7bf8389640 1 -- start start 2023-12-12T09:53:43.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.952+0000 7f7bf8389640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7bf015af50 0x7f7bf006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.952+0000 7f7bf8389640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf0071cd0 con 0x7f7bf00ce7e0 2023-12-12T09:53:43.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.952+0000 7f7bf8389640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf006c6b0 con 0x7f7bf006a460 2023-12-12T09:53:43.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.952+0000 7f7bf8389640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf00717e0 con 0x7f7bf015af50 2023-12-12T09:53:43.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.953+0000 7f7bf6125640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7bf015af50 0x7f7bf006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:43.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.953+0000 7f7bf5924640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f7bf00ce7e0 0x7f7bf015ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59410/0 (socket says 172.21.15.72:59410) 2023-12-12T09:53:43.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.953+0000 7f7bf5924640 1 -- 172.21.15.72:0/1862098838 learned_addr learned my addr 172.21.15.72:0/1862098838 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:43.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.953+0000 7f7bf6125640 1 -- 172.21.15.72:0/1862098838 >> v1:172.21.15.113:6789/0 conn(0x7f7bf006a460 legacy=0x7f7bf00ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:43.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.953+0000 7f7bf6125640 1 -- 172.21.15.72:0/1862098838 >> v1:172.21.15.72:6789/0 conn(0x7f7bf00ce7e0 legacy=0x7f7bf015ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.953+0000 7f7bf5123640 1 -- 172.21.15.72:0/1862098838 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7765221 0 0) 0x7f7bf0071cd0 con 0x7f7bf00ce7e0 2023-12-12T09:53:43.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.953+0000 7f7bf6125640 1 -- 172.21.15.72:0/1862098838 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf00719f0 con 0x7f7bf015af50 2023-12-12T09:53:43.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.954+0000 7f7bf6125640 1 --2- 172.21.15.72:0/1862098838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf015af50 0x7f7bf006c170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0097a0 tx=0x7f7bec02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da26a69bd6716d5a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:43.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf5123640 1 -- 172.21.15.72:0/1862098838 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec02f050 con 0x7f7bf015af50 2023-12-12T09:53:43.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf5123640 1 -- 172.21.15.72:0/1862098838 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec031700 con 0x7f7bf015af50 2023-12-12T09:53:43.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf5123640 1 -- 172.21.15.72:0/1862098838 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec031a90 con 0x7f7bf015af50 2023-12-12T09:53:43.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf8389640 1 -- 172.21.15.72:0/1862098838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf015af50 msgr2=0x7f7bf006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf8389640 1 --2- 172.21.15.72:0/1862098838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf015af50 0x7f7bf006c170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0097a0 tx=0x7f7bec02b7a0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf8389640 1 -- 172.21.15.72:0/1862098838 shutdown_connections 2023-12-12T09:53:43.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf8389640 1 --2- 172.21.15.72:0/1862098838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf015af50 0x7f7bf006c170 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf8389640 1 -- 172.21.15.72:0/1862098838 >> 172.21.15.72:0/1862098838 conn(0x7f7bf00cee70 msgr2=0x7f7bf00cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:43.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf8389640 1 -- 172.21.15.72:0/1862098838 shutdown_connections 2023-12-12T09:53:43.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.955+0000 7f7bf8389640 1 -- 172.21.15.72:0/1862098838 wait complete. 2023-12-12T09:53:43.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.957+0000 7f7bf8389640 1 Processor -- start 2023-12-12T09:53:43.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.957+0000 7f7bf8389640 1 -- start start 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.958+0000 7f7bf8389640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf006a460 0x7f7bf009b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.958+0000 7f7bf8389640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bf00ce7e0 0x7f7bf00c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.958+0000 7f7bf8389640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf015af50 0x7f7bf0147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.958+0000 7f7bf8389640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00c2890 con 0x7f7bf006a460 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.958+0000 7f7bf8389640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf009bdd0 con 0x7f7bf015af50 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.958+0000 7f7bf8389640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf009c0b0 con 0x7f7bf00ce7e0 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.958+0000 7f7bf6125640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf006a460 0x7f7bf009b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6125640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf006a460 0x7f7bf009b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52972/0 (socket says 172.21.15.72:52972) 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6125640 1 -- 172.21.15.72:0/1542240889 learned_addr learned my addr 172.21.15.72:0/1542240889 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:43.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6125640 1 -- 172.21.15.72:0/1542240889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bf00ce7e0 msgr2=0x7f7bf00c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6125640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bf00ce7e0 0x7f7bf00c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6125640 1 -- 172.21.15.72:0/1542240889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf015af50 msgr2=0x7f7bf0147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6926640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf015af50 0x7f7bf0147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:43.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6125640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf015af50 0x7f7bf0147cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6125640 1 -- 172.21.15.72:0/1542240889 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec009400 con 0x7f7bf006a460 2023-12-12T09:53:43.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.959+0000 7f7bf6125640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf006a460 0x7f7bf009b580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0097a0 tx=0x7f7bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:43.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.960+0000 7f7be6ffd640 1 -- 172.21.15.72:0/1542240889 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec033c30 con 0x7f7bf006a460 2023-12-12T09:53:43.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.960+0000 7f7be6ffd640 1 -- 172.21.15.72:0/1542240889 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec033d90 con 0x7f7bf006a460 2023-12-12T09:53:43.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.960+0000 7f7be6ffd640 1 -- 172.21.15.72:0/1542240889 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec03d6b0 con 0x7f7bf006a460 2023-12-12T09:53:43.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.960+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf009c390 con 0x7f7bf006a460 2023-12-12T09:53:43.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.960+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf009bac0 con 0x7f7bf006a460 2023-12-12T09:53:43.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.961+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf0074450 con 0x7f7bf006a460 2023-12-12T09:53:43.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.962+0000 7f7be6ffd640 1 -- 172.21.15.72:0/1542240889 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bec03a040 con 0x7f7bf006a460 2023-12-12T09:53:43.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.963+0000 7f7be6ffd640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bd406ae40 0x7f7bd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:43.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.963+0000 7f7be6ffd640 1 -- 172.21.15.72:0/1542240889 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7bec0ab440 con 0x7f7bf006a460 2023-12-12T09:53:43.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.964+0000 7f7bf5924640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bd406ae40 0x7f7bd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:43.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.964+0000 7f7bf5924640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bd406ae40 0x7f7bd406d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7be0007fd0 tx=0x7f7be0007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:43.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:43.968+0000 7f7be6ffd640 1 -- 172.21.15.72:0/1542240889 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bec076660 con 0x7f7bf006a460 2023-12-12T09:53:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.159+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bf00ce1d0 con 0x7f7bf006a460 2023-12-12T09:53:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.160+0000 7f7be6ffd640 1 -- 172.21.15.72:0/1542240889 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bec079c80 con 0x7f7bf006a460 2023-12-12T09:53:44.163 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:53:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bd406ae40 msgr2=0x7f7bd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bd406ae40 0x7f7bd406d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7be0007fd0 tx=0x7f7be0007960 comp rx=0 tx=0).stop 2023-12-12T09:53:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf006a460 msgr2=0x7f7bf009b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf006a460 0x7f7bf009b580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0097a0 tx=0x7f7bec0046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 shutdown_connections 2023-12-12T09:53:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bd406ae40 0x7f7bd406d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bf015af50 0x7f7bf0147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bf00ce7e0 0x7f7bf00c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 --2- 172.21.15.72:0/1542240889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bf006a460 0x7f7bf009b580 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.162+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 >> 172.21.15.72:0/1542240889 conn(0x7f7bf00cee70 msgr2=0x7f7bf015c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.163+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 shutdown_connections 2023-12-12T09:53:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.163+0000 7f7bf8389640 1 -- 172.21.15.72:0/1542240889 wait complete. 2023-12-12T09:53:44.183 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-12T09:53:44.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.372+0000 7f5c17341640 1 Processor -- start 2023-12-12T09:53:44.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.372+0000 7f5c17341640 1 -- start start 2023-12-12T09:53:44.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.374+0000 7f5c17341640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c101609b0 0x7f5c100d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.374+0000 7f5c17341640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1015dd60 con 0x7f5c100d1220 2023-12-12T09:53:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.374+0000 7f5c17341640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1015e040 con 0x7f5c100d0230 2023-12-12T09:53:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.374+0000 7f5c17341640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10160d90 con 0x7f5c101609b0 2023-12-12T09:53:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.374+0000 7f5c148dc640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f5c100d1220 0x7f5c100cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59414/0 (socket says 172.21.15.72:59414) 2023-12-12T09:53:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.374+0000 7f5c148dc640 1 -- 172.21.15.72:0/2835541766 learned_addr learned my addr 172.21.15.72:0/2835541766 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c150dd640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c101609b0 0x7f5c100d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269099804 0 0) 0x7f5c1015dd60 con 0x7f5c100d1220 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bfc003520 con 0x7f5c100d1220 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3869334909 0 0) 0x7f5bfc003520 con 0x7f5c100d1220 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c101609b0 msgr2=0x7f5c100d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c101609b0 0x7f5c100d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 >> v1:172.21.15.113:6789/0 conn(0x7f5c100d0230 legacy=0x7f5c1015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1015e320 con 0x7f5c100d1220 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f5c00001670 con 0x7f5c100d1220 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 >> v1:172.21.15.72:6789/0 conn(0x7f5c100d1220 legacy=0x7f5c100cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bfc0039a0 0x7f5bfc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.375+0000 7f5c07fff640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bfc0047c0 0x7f5bfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c07fff640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c101609b0 0x7f5bfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10160d90 con 0x7f5bfc0039a0 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc00bab0 con 0x7f5bfc0047c0 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc00bc70 con 0x7f5c101609b0 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c150dd640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bfc0039a0 0x7f5bfc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c150dd640 1 -- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c101609b0 msgr2=0x7f5bfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c150dd640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c101609b0 0x7f5bfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c150dd640 1 -- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bfc0047c0 msgr2=0x7f5bfc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c150dd640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bfc0047c0 0x7f5bfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.376+0000 7f5c150dd640 1 -- 172.21.15.72:0/2835541766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc003520 con 0x7f5bfc0039a0 2023-12-12T09:53:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.377+0000 7f5c150dd640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bfc0039a0 0x7f5bfc003dc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00ba30 tx=0x7f5bfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56aafe028d6bdf5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.377+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c032680 con 0x7f5bfc0039a0 2023-12-12T09:53:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c0327e0 con 0x7f5bfc0039a0 2023-12-12T09:53:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c07fff640 1 -- 172.21.15.72:0/2835541766 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c032b70 con 0x7f5bfc0039a0 2023-12-12T09:53:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c17341640 1 -- 172.21.15.72:0/2835541766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bfc0039a0 msgr2=0x7f5bfc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c17341640 1 --2- 172.21.15.72:0/2835541766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bfc0039a0 0x7f5bfc003dc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00ba30 tx=0x7f5bfc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c158de640 1 -- 172.21.15.72:0/2835541766 reap_dead start 2023-12-12T09:53:44.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c17341640 1 -- 172.21.15.72:0/2835541766 shutdown_connections 2023-12-12T09:53:44.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c17341640 1 -- 172.21.15.72:0/2835541766 >> 172.21.15.72:0/2835541766 conn(0x7f5c1009c450 msgr2=0x7f5c100d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:44.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c17341640 1 -- 172.21.15.72:0/2835541766 shutdown_connections 2023-12-12T09:53:44.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.378+0000 7f5c17341640 1 -- 172.21.15.72:0/2835541766 wait complete. 2023-12-12T09:53:44.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.380+0000 7f5c17341640 1 Processor -- start 2023-12-12T09:53:44.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.380+0000 7f5c17341640 1 -- start start 2023-12-12T09:53:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.381+0000 7f5c17341640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bfc0039a0 0x7f5c1009d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.381+0000 7f5c17341640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c100c87c0 0x7f5c100cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.381+0000 7f5c17341640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c100ca1e0 0x7f5c100cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.381+0000 7f5c17341640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100cb3b0 con 0x7f5c100c87c0 2023-12-12T09:53:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.381+0000 7f5c17341640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100ca7a0 con 0x7f5c100ca1e0 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.381+0000 7f5c17341640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100caa80 con 0x7f5bfc0039a0 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c100c87c0 0x7f5c100cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c100c87c0 0x7f5c100cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52998/0 (socket says 172.21.15.72:52998) 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 -- 172.21.15.72:0/300326225 learned_addr learned my addr 172.21.15.72:0/300326225 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c150dd640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bfc0039a0 0x7f5c1009d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c158de640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c100ca1e0 0x7f5c100cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 -- 172.21.15.72:0/300326225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bfc0039a0 msgr2=0x7f5c1009d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bfc0039a0 0x7f5c1009d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 -- 172.21.15.72:0/300326225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c100ca1e0 msgr2=0x7f5c100cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c100ca1e0 0x7f5c100cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 -- 172.21.15.72:0/300326225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100cad60 con 0x7f5c100c87c0 2023-12-12T09:53:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.382+0000 7f5c148dc640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c100c87c0 0x7f5c100cb840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000c700 tx=0x7f5c0000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.383+0000 7f5c05ffb640 1 -- 172.21.15.72:0/300326225 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000074e0 con 0x7f5c100c87c0 2023-12-12T09:53:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.383+0000 7f5c05ffb640 1 -- 172.21.15.72:0/300326225 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00007640 con 0x7f5c100c87c0 2023-12-12T09:53:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.383+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c100cb040 con 0x7f5c100c87c0 2023-12-12T09:53:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.383+0000 7f5c05ffb640 1 -- 172.21.15.72:0/300326225 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000049e0 con 0x7f5c100c87c0 2023-12-12T09:53:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.383+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1009d670 con 0x7f5c100c87c0 2023-12-12T09:53:44.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.384+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c100745d0 con 0x7f5c100c87c0 2023-12-12T09:53:44.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.386+0000 7f5c05ffb640 1 -- 172.21.15.72:0/300326225 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c00004b40 con 0x7f5c100c87c0 2023-12-12T09:53:44.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.386+0000 7f5c05ffb640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5bf406ae70 0x7f5bf406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.387+0000 7f5c05ffb640 1 -- 172.21.15.72:0/300326225 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5c00088b70 con 0x7f5c100c87c0 2023-12-12T09:53:44.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.387+0000 7f5c150dd640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5bf406ae70 0x7f5bf406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.387+0000 7f5c150dd640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5bf406ae70 0x7f5bf406d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00ba00 tx=0x7f5c0c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:44.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.394+0000 7f5c05ffb640 1 -- 172.21.15.72:0/300326225 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c00053d80 con 0x7f5c100c87c0 2023-12-12T09:53:44.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.587+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5c100d3820 con 0x7f5c100c87c0 2023-12-12T09:53:44.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.588+0000 7f5c05ffb640 1 -- 172.21.15.72:0/300326225 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5c00018070 con 0x7f5c100c87c0 2023-12-12T09:53:44.590 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:44.590 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2023-12-12T09:53:44.590 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:53:44.590 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 1, 2023-12-12T09:53:44.591 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": ".mgr" 2023-12-12T09:53:44.591 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:53:44.591 INFO:teuthology.orchestra.run.smithi072.stdout:] 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5bf406ae70 msgr2=0x7f5bf406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5bf406ae70 0x7f5bf406d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00ba00 tx=0x7f5c0c03d040 comp rx=0 tx=0).stop 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c100c87c0 msgr2=0x7f5c100cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c100c87c0 0x7f5c100cb840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000c700 tx=0x7f5c0000cbc0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 shutdown_connections 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5bf406ae70 0x7f5bf406d330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c100ca1e0 0x7f5c100cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c100c87c0 0x7f5c100cb840 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 --2- 172.21.15.72:0/300326225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bfc0039a0 0x7f5c1009d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 >> 172.21.15.72:0/300326225 conn(0x7f5c1009c450 msgr2=0x7f5c1015f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:44.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 shutdown_connections 2023-12-12T09:53:44.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.591+0000 7f5c17341640 1 -- 172.21.15.72:0/300326225 wait complete. 2023-12-12T09:53:44.612 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-12T09:53:44.612 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-12T09:53:44.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.825+0000 7f710f06d640 1 Processor -- start 2023-12-12T09:53:44.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.826+0000 7f710f06d640 1 -- start start 2023-12-12T09:53:44.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.827+0000 7f710f06d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71080d15b0 0x7f71080d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.827+0000 7f710f06d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080ce5d0 con 0x7f71080d1990 2023-12-12T09:53:44.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.827+0000 7f710f06d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080ce870 con 0x7f710815d4a0 2023-12-12T09:53:44.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.827+0000 7f710f06d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710815d8f0 con 0x7f71080d15b0 2023-12-12T09:53:44.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.827+0000 7f710d86a640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f71080d1990 0x7f710815cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59422/0 (socket says 172.21.15.72:59422) 2023-12-12T09:53:44.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.827+0000 7f710d86a640 1 -- 172.21.15.72:0/3437822541 learned_addr learned my addr 172.21.15.72:0/3437822541 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:44.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710e06b640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71080d15b0 0x7f71080d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757967123 0 0) 0x7f71080ce5d0 con 0x7f71080d1990 2023-12-12T09:53:44.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70f4003520 con 0x7f71080d1990 2023-12-12T09:53:44.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 127587200 0 0) 0x7f70f4003520 con 0x7f71080d1990 2023-12-12T09:53:44.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71080d15b0 msgr2=0x7f71080d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:44.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71080d15b0 0x7f71080d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 >> v1:172.21.15.113:6789/0 conn(0x7f710815d4a0 legacy=0x7f71080afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080ceb50 con 0x7f71080d1990 2023-12-12T09:53:44.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f70f8001670 con 0x7f71080d1990 2023-12-12T09:53:44.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 >> v1:172.21.15.72:6789/0 conn(0x7f71080d1990 legacy=0x7f710815cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40039a0 0x7f70f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70f40047c0 0x7f70f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71080d15b0 0x7f70f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40054b0 con 0x7f70f40047c0 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f4005620 con 0x7f70f40039a0 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.828+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f400bec0 con 0x7f71080d15b0 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.829+0000 7f710d86a640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70f40047c0 0x7f70f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.829+0000 7f710e06b640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40039a0 0x7f70f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.829+0000 7f710e86c640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71080d15b0 0x7f70f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.829+0000 7f710d86a640 1 -- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71080d15b0 msgr2=0x7f70f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:44.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.829+0000 7f710d86a640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71080d15b0 0x7f70f400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.829+0000 7f710d86a640 1 -- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40039a0 msgr2=0x7f70f4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:44.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.829+0000 7f710d86a640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40039a0 0x7f70f4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.829+0000 7f710d86a640 1 -- 172.21.15.72:0/3437822541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f4003520 con 0x7f70f40047c0 2023-12-12T09:53:44.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.830+0000 7f710d86a640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70f40047c0 0x7f70f4004c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c240 tx=0x7f70f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6859ea702ec731 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:44.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.830+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8005ad0 con 0x7f70f40047c0 2023-12-12T09:53:44.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.830+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8005c30 con 0x7f70f40047c0 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.830+0000 7f710d069640 1 -- 172.21.15.72:0/3437822541 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f802c8b0 con 0x7f70f40047c0 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.831+0000 7f710f06d640 1 -- 172.21.15.72:0/3437822541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70f40047c0 msgr2=0x7f70f4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.831+0000 7f710f06d640 1 --2- 172.21.15.72:0/3437822541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70f40047c0 0x7f70f4004c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c240 tx=0x7f70f40057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.831+0000 7f710e86c640 1 -- 172.21.15.72:0/3437822541 reap_dead start 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.831+0000 7f710f06d640 1 -- 172.21.15.72:0/3437822541 shutdown_connections 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.831+0000 7f710f06d640 1 -- 172.21.15.72:0/3437822541 >> 172.21.15.72:0/3437822541 conn(0x7f71080d2f10 msgr2=0x7f71080d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.831+0000 7f710f06d640 1 -- 172.21.15.72:0/3437822541 shutdown_connections 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.831+0000 7f710f06d640 1 -- 172.21.15.72:0/3437822541 wait complete. 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.833+0000 7f710f06d640 1 Processor -- start 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.833+0000 7f710f06d640 1 -- start start 2023-12-12T09:53:44.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710f06d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f40047c0 0x7f71080c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710f06d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f710815c940 0x7f710815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710f06d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71080b5dd0 0x7f71080b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710f06d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080b6790 con 0x7f710815c940 2023-12-12T09:53:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710f06d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108149f80 con 0x7f71080b5dd0 2023-12-12T09:53:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710f06d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080c9360 con 0x7f70f40047c0 2023-12-12T09:53:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710d86a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f710815c940 0x7f710815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710d86a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f710815c940 0x7f710815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53012/0 (socket says 172.21.15.72:53012) 2023-12-12T09:53:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.834+0000 7f710d86a640 1 -- 172.21.15.72:0/2546055777 learned_addr learned my addr 172.21.15.72:0/2546055777 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710e06b640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f40047c0 0x7f71080c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710e86c640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71080b5dd0 0x7f71080b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710d86a640 1 -- 172.21.15.72:0/2546055777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f40047c0 msgr2=0x7f71080c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710d86a640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f40047c0 0x7f71080c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710d86a640 1 -- 172.21.15.72:0/2546055777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71080b5dd0 msgr2=0x7f71080b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710d86a640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71080b5dd0 0x7f71080b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:44.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710d86a640 1 -- 172.21.15.72:0/2546055777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710814a220 con 0x7f710815c940 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710d86a640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f710815c940 0x7f710815cdc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f70f802c450 tx=0x7f70f8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f70feffd640 1 -- 172.21.15.72:0/2546055777 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8039070 con 0x7f710815c940 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f70feffd640 1 -- 172.21.15.72:0/2546055777 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f802dd50 con 0x7f710815c940 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710814a500 con 0x7f710815c940 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f70feffd640 1 -- 172.21.15.72:0/2546055777 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f803c040 con 0x7f710815c940 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.835+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710814aac0 con 0x7f710815c940 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.836+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71080d2980 con 0x7f710815c940 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.838+0000 7f70feffd640 1 -- 172.21.15.72:0/2546055777 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70f8032070 con 0x7f710815c940 2023-12-12T09:53:44.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.839+0000 7f70feffd640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f70ec06ae40 0x7f70ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.839+0000 7f70feffd640 1 -- 172.21.15.72:0/2546055777 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f70f80aa8c0 con 0x7f710815c940 2023-12-12T09:53:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.839+0000 7f710e06b640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f70ec06ae40 0x7f70ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.839+0000 7f710e06b640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f70ec06ae40 0x7f70ec06d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7104006fd0 tx=0x7f7104008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:44.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:44.843+0000 7f70feffd640 1 -- 172.21.15.72:0/2546055777 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70f8075c10 con 0x7f710815c940 2023-12-12T09:53:45.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.031+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7108074450 con 0x7f710815c940 2023-12-12T09:53:45.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.775+0000 7f70feffd640 1 -- 172.21.15.72:0/2546055777 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f70f8046020 con 0x7f710815c940 2023-12-12T09:53:45.778 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' created 2023-12-12T09:53:45.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f70ec06ae40 msgr2=0x7f70ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f70ec06ae40 0x7f70ec06d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7104006fd0 tx=0x7f7104008040 comp rx=0 tx=0).stop 2023-12-12T09:53:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f710815c940 msgr2=0x7f710815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f710815c940 0x7f710815cdc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f70f802c450 tx=0x7f70f8001a00 comp rx=0 tx=0).stop 2023-12-12T09:53:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 shutdown_connections 2023-12-12T09:53:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f70ec06ae40 0x7f70ec06d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71080b5dd0 0x7f71080b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f710815c940 0x7f710815cdc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:45.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 --2- 172.21.15.72:0/2546055777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f40047c0 0x7f71080c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:45.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.779+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 >> 172.21.15.72:0/2546055777 conn(0x7f71080d2f10 msgr2=0x7f710806afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:45.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.780+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 shutdown_connections 2023-12-12T09:53:45.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:45.780+0000 7f710f06d640 1 -- 172.21.15.72:0/2546055777 wait complete. 2023-12-12T09:53:45.808 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-12T09:53:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.010+0000 7fea6759e640 1 Processor -- start 2023-12-12T09:53:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.010+0000 7fea6759e640 1 -- start start 2023-12-12T09:53:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.012+0000 7fea6759e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea6815d210 0x7fea680d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.012+0000 7fea6759e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea680d1c60 0x7fea681672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.012+0000 7fea6759e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea680cdb80 con 0x7fea680d3bc0 2023-12-12T09:53:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.012+0000 7fea6759e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea680b48e0 con 0x7fea6815d210 2023-12-12T09:53:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.012+0000 7fea6759e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea680cddb0 con 0x7fea680d1c60 2023-12-12T09:53:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.012+0000 7fea5ffff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fea680d3bc0 0x7fea680d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59902/0 (socket says 172.21.15.72:59902) 2023-12-12T09:53:46.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.012+0000 7fea5ffff640 1 -- 172.21.15.72:0/1327801448 learned_addr learned my addr 172.21.15.72:0/1327801448 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:46.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea6659c640 1 --2- 172.21.15.72:0/1327801448 >> v2:172.21.15.113:3300/0 conn(0x7fea6815d210 0x7fea680d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea66d9d640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea680d1c60 0x7fea681672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035421618 0 0) 0x7fea680cdb80 con 0x7fea680d3bc0 2023-12-12T09:53:46.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea54003520 con 0x7fea680d3bc0 2023-12-12T09:53:46.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3770027216 0 0) 0x7fea54003520 con 0x7fea680d3bc0 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea680d1c60 msgr2=0x7fea681672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea680d1c60 0x7fea681672d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 >> v2:172.21.15.113:3300/0 conn(0x7fea6815d210 msgr2=0x7fea680d3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 --2- 172.21.15.72:0/1327801448 >> v2:172.21.15.113:3300/0 conn(0x7fea6815d210 0x7fea680d3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea680ce050 con 0x7fea680d3bc0 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fea50001670 con 0x7fea680d3bc0 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 >> v1:172.21.15.72:6789/0 conn(0x7fea680d3bc0 legacy=0x7fea680d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea680d1c60 0x7fea54003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea54003ef0 0x7fea540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea54004d00 0x7fea54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea680b48e0 con 0x7fea54004d00 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea680cddb0 con 0x7fea54003ef0 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.013+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5400b980 con 0x7fea680d1c60 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.014+0000 7fea66d9d640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea54004d00 0x7fea54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.014+0000 7fea6659c640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea680d1c60 0x7fea54003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.014+0000 7fea5ffff640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea54003ef0 0x7fea540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.014+0000 7fea66d9d640 1 -- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea680d1c60 msgr2=0x7fea54003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.014+0000 7fea66d9d640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea680d1c60 0x7fea54003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.014+0000 7fea66d9d640 1 -- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea54003ef0 msgr2=0x7fea540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.014+0000 7fea66d9d640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea54003ef0 0x7fea540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.014+0000 7fea66d9d640 1 -- 172.21.15.72:0/1327801448 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea54003520 con 0x7fea54004d00 2023-12-12T09:53:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.015+0000 7fea66d9d640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea54004d00 0x7fea54005180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fea680cd670 tx=0x7fea540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455af53f17cb6ef3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6002cb90 con 0x7fea54004d00 2023-12-12T09:53:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6000f040 con 0x7fea54004d00 2023-12-12T09:53:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea65d9b640 1 -- 172.21.15.72:0/1327801448 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60034440 con 0x7fea54004d00 2023-12-12T09:53:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea6759e640 1 -- 172.21.15.72:0/1327801448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea54004d00 msgr2=0x7fea54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea6759e640 1 --2- 172.21.15.72:0/1327801448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea54004d00 0x7fea54005180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fea680cd670 tx=0x7fea540057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea66d9d640 1 -- 172.21.15.72:0/1327801448 reap_dead start 2023-12-12T09:53:46.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea6759e640 1 -- 172.21.15.72:0/1327801448 shutdown_connections 2023-12-12T09:53:46.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea6759e640 1 -- 172.21.15.72:0/1327801448 >> 172.21.15.72:0/1327801448 conn(0x7fea6809c250 msgr2=0x7fea680d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:46.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea6759e640 1 -- 172.21.15.72:0/1327801448 shutdown_connections 2023-12-12T09:53:46.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.016+0000 7fea6759e640 1 -- 172.21.15.72:0/1327801448 wait complete. 2023-12-12T09:53:46.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.018+0000 7fea6759e640 1 Processor -- start 2023-12-12T09:53:46.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.018+0000 7fea6759e640 1 -- start start 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.019+0000 7fea6759e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea54004d00 0x7fea6815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.019+0000 7fea6759e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea68151580 0x7fea68160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.019+0000 7fea6759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea681f31f0 0x7fea6806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.019+0000 7fea6759e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6815a8e0 con 0x7fea68151580 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.019+0000 7fea6759e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea68151a00 con 0x7fea681f31f0 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.019+0000 7fea6759e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea681609d0 con 0x7fea54004d00 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.019+0000 7fea5ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea68151580 0x7fea68160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea5ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea68151580 0x7fea68160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52094/0 (socket says 172.21.15.72:52094) 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea5ffff640 1 -- 172.21.15.72:0/844905379 learned_addr learned my addr 172.21.15.72:0/844905379 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea6659c640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea54004d00 0x7fea6815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea66d9d640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea681f31f0 0x7fea6806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea5ffff640 1 -- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea54004d00 msgr2=0x7fea6815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea5ffff640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea54004d00 0x7fea6815d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea5ffff640 1 -- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea681f31f0 msgr2=0x7fea6806b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea5ffff640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea681f31f0 0x7fea6806b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea5ffff640 1 -- 172.21.15.72:0/844905379 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea680cb950 con 0x7fea68151580 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea66d9d640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea681f31f0 0x7fea6806b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.020+0000 7fea5ffff640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea68151580 0x7fea68160490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fea5000e8a0 tx=0x7fea5000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.021+0000 7fea5effd640 1 -- 172.21.15.72:0/844905379 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000c870 con 0x7fea68151580 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.021+0000 7fea5effd640 1 -- 172.21.15.72:0/844905379 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5000c9f0 con 0x7fea68151580 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.021+0000 7fea5effd640 1 -- 172.21.15.72:0/844905379 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000cda0 con 0x7fea68151580 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.021+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea680cbc30 con 0x7fea68151580 2023-12-12T09:53:46.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.021+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea680cc1f0 con 0x7fea68151580 2023-12-12T09:53:46.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.022+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea68074450 con 0x7fea68151580 2023-12-12T09:53:46.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.023+0000 7fea5effd640 1 -- 172.21.15.72:0/844905379 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea5001d460 con 0x7fea68151580 2023-12-12T09:53:46.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.024+0000 7fea5effd640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea4806ae40 0x7fea4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:46.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.024+0000 7fea5effd640 1 -- 172.21.15.72:0/844905379 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fea500895f0 con 0x7fea68151580 2023-12-12T09:53:46.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.024+0000 7fea6659c640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea4806ae40 0x7fea4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:46.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.025+0000 7fea6659c640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea4806ae40 0x7fea4806d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fea580099d0 tx=0x7fea58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:46.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.029+0000 7fea5effd640 1 -- 172.21.15.72:0/844905379 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea50054730 con 0x7fea68151580 2023-12-12T09:53:46.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.219+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fea680cc3b0 con 0x7fea68151580 2023-12-12T09:53:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.792+0000 7fea5effd640 1 -- 172.21.15.72:0/844905379 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7fea50057d50 con 0x7fea68151580 2023-12-12T09:53:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' created 2023-12-12T09:53:46.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.796+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea4806ae40 msgr2=0x7fea4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:46.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.796+0000 7fea6759e640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea4806ae40 0x7fea4806d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fea580099d0 tx=0x7fea58008040 comp rx=0 tx=0).stop 2023-12-12T09:53:46.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.796+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea68151580 msgr2=0x7fea68160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:46.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.796+0000 7fea6759e640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea68151580 0x7fea68160490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fea5000e8a0 tx=0x7fea5000ed60 comp rx=0 tx=0).stop 2023-12-12T09:53:46.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.798+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 shutdown_connections 2023-12-12T09:53:46.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.799+0000 7fea6759e640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fea4806ae40 0x7fea4806d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.799+0000 7fea6759e640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea681f31f0 0x7fea6806b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.799+0000 7fea6759e640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea68151580 0x7fea68160490 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.799+0000 7fea6759e640 1 --2- 172.21.15.72:0/844905379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea54004d00 0x7fea6815d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:46.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.799+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 >> 172.21.15.72:0/844905379 conn(0x7fea6809c250 msgr2=0x7fea680b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:46.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.801+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 shutdown_connections 2023-12-12T09:53:46.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:46.801+0000 7fea6759e640 1 -- 172.21.15.72:0/844905379 wait complete. 2023-12-12T09:53:46.835 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-12T09:53:47.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.019+0000 7f0cc961e640 1 Processor -- start 2023-12-12T09:53:47.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.019+0000 7f0cc961e640 1 -- start start 2023-12-12T09:53:47.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.020+0000 7f0cc961e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40b46a0 0x7f0cc40b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.020+0000 7f0cc961e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cc40b36a0 0x7f0cc40cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cc961e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40d0af0 con 0x7f0cc4160b40 2023-12-12T09:53:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cc961e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc4160f20 con 0x7f0cc40b36a0 2023-12-12T09:53:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cc961e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40d1520 con 0x7f0cc40b46a0 2023-12-12T09:53:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cc2ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f0cc4160b40 0x7f0cc40b2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59908/0 (socket says 172.21.15.72:59908) 2023-12-12T09:53:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cc2ffd640 1 -- 172.21.15.72:0/1899890218 learned_addr learned my addr 172.21.15.72:0/1899890218 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cc37fe640 1 --2- 172.21.15.72:0/1899890218 >> v2:172.21.15.113:3300/0 conn(0x7f0cc40b36a0 0x7f0cc40cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cba7fc640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40b46a0 0x7f0cc40b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739894891 0 0) 0x7f0cc40d0af0 con 0x7f0cc4160b40 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.021+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cb0003520 con 0x7f0cc4160b40 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2574002116 0 0) 0x7f0cb0003520 con 0x7f0cc4160b40 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40b46a0 msgr2=0x7f0cc40b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40b46a0 0x7f0cc40b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 >> v2:172.21.15.113:3300/0 conn(0x7f0cc40b36a0 msgr2=0x7f0cc40cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 --2- 172.21.15.72:0/1899890218 >> v2:172.21.15.113:3300/0 conn(0x7f0cc40b36a0 0x7f0cc40cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40cf040 con 0x7f0cc4160b40 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f0cb40019f0 con 0x7f0cc4160b40 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 >> v1:172.21.15.72:6789/0 conn(0x7f0cc4160b40 legacy=0x7f0cc40b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb00047c0 0x7f0cb0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40b46a0 0x7f0cb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4160f20 con 0x7f0cb0003970 2023-12-12T09:53:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40d1520 con 0x7f0cb00047c0 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb000b980 con 0x7f0cc40b46a0 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.022+0000 7f0cc2ffd640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.023+0000 7f0cc37fe640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40b46a0 0x7f0cb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.023+0000 7f0cc2ffd640 1 -- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40b46a0 msgr2=0x7f0cb0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.023+0000 7f0cc2ffd640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40b46a0 0x7f0cb0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.023+0000 7f0cc2ffd640 1 -- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb00047c0 msgr2=0x7f0cb0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.023+0000 7f0cc2ffd640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb00047c0 0x7f0cb0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.023+0000 7f0cc2ffd640 1 -- 172.21.15.72:0/1899890218 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb0003520 con 0x7f0cb0003970 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.024+0000 7f0cc2ffd640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cb0003d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400a830 tx=0x7f0cb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c1fe385b2b7049 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.024+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4034630 con 0x7f0cb0003970 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.024+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb402f070 con 0x7f0cb0003970 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.024+0000 7f0cc27fc640 1 -- 172.21.15.72:0/1899890218 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40349d0 con 0x7f0cb0003970 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.024+0000 7f0cc961e640 1 -- 172.21.15.72:0/1899890218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 msgr2=0x7f0cb0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.024+0000 7f0cc961e640 1 --2- 172.21.15.72:0/1899890218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cb0003d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400a830 tx=0x7f0cb00057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.025+0000 7f0cc37fe640 1 -- 172.21.15.72:0/1899890218 reap_dead start 2023-12-12T09:53:47.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.025+0000 7f0cc961e640 1 -- 172.21.15.72:0/1899890218 shutdown_connections 2023-12-12T09:53:47.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.025+0000 7f0cc961e640 1 -- 172.21.15.72:0/1899890218 >> 172.21.15.72:0/1899890218 conn(0x7f0cc409c420 msgr2=0x7f0cc40d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:47.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.025+0000 7f0cc961e640 1 -- 172.21.15.72:0/1899890218 shutdown_connections 2023-12-12T09:53:47.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.025+0000 7f0cc961e640 1 -- 172.21.15.72:0/1899890218 wait complete. 2023-12-12T09:53:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.026+0000 7f0cc961e640 1 Processor -- start 2023-12-12T09:53:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.027+0000 7f0cc961e640 1 -- start start 2023-12-12T09:53:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.027+0000 7f0cc961e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cc414f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.027+0000 7f0cc961e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc40cbce0 0x7f0cc40ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc961e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40ca700 0x7f0cc40b7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc961e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40cac40 con 0x7f0cb0003970 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc961e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40caf20 con 0x7f0cc40cbce0 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc961e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40cb200 con 0x7f0cc40ca700 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cc414f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cc414f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52112/0 (socket says 172.21.15.72:52112) 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc2ffd640 1 -- 172.21.15.72:0/4285139068 learned_addr learned my addr 172.21.15.72:0/4285139068 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc37fe640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40ca700 0x7f0cc40b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc2ffd640 1 -- 172.21.15.72:0/4285139068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40ca700 msgr2=0x7f0cc40b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cba7fc640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc40cbce0 0x7f0cc40ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:47.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc2ffd640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40ca700 0x7f0cc40b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc2ffd640 1 -- 172.21.15.72:0/4285139068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc40cbce0 msgr2=0x7f0cc40ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc2ffd640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc40cbce0 0x7f0cc40ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.028+0000 7f0cc2ffd640 1 -- 172.21.15.72:0/4285139068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40cb4e0 con 0x7f0cb0003970 2023-12-12T09:53:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.029+0000 7f0cc2ffd640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cc414f420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0cb402cb70 tx=0x7f0cc40b2400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.029+0000 7f0cbbfff640 1 -- 172.21.15.72:0/4285139068 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4035750 con 0x7f0cb0003970 2023-12-12T09:53:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.029+0000 7f0cbbfff640 1 -- 172.21.15.72:0/4285139068 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb40358b0 con 0x7f0cb0003970 2023-12-12T09:53:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.029+0000 7f0cbbfff640 1 -- 172.21.15.72:0/4285139068 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4035c20 con 0x7f0cb0003970 2023-12-12T09:53:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.029+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc40cb7c0 con 0x7f0cb0003970 2023-12-12T09:53:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.029+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc41587e0 con 0x7f0cb0003970 2023-12-12T09:53:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.030+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc4074450 con 0x7f0cb0003970 2023-12-12T09:53:47.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.032+0000 7f0cbbfff640 1 -- 172.21.15.72:0/4285139068 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0cb4031030 con 0x7f0cb0003970 2023-12-12T09:53:47.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.033+0000 7f0cbbfff640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0ca406ae70 0x7f0ca406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:47.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.033+0000 7f0cbbfff640 1 -- 172.21.15.72:0/4285139068 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f0cb40ac180 con 0x7f0cb0003970 2023-12-12T09:53:47.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.034+0000 7f0cba7fc640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0ca406ae70 0x7f0ca406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:47.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.034+0000 7f0cba7fc640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0ca406ae70 0x7f0ca406d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cc41585c0 tx=0x7f0cac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:47.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.037+0000 7f0cbbfff640 1 -- 172.21.15.72:0/4285139068 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cb407ac30 con 0x7f0cb0003970 2023-12-12T09:53:47.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.251+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0cc406b960 con 0x7f0cb0003970 2023-12-12T09:53:47.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.817+0000 7f0cbbfff640 1 -- 172.21.15.72:0/4285139068 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0cb404d020 con 0x7f0cb0003970 2023-12-12T09:53:47.820 INFO:teuthology.orchestra.run.smithi072.stderr:new fs with metadata pool 5 and data pool 6 2023-12-12T09:53:47.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.825+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0ca406ae70 msgr2=0x7f0ca406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:47.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.825+0000 7f0cc961e640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0ca406ae70 0x7f0ca406d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cc41585c0 tx=0x7f0cac008040 comp rx=0 tx=0).stop 2023-12-12T09:53:47.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.825+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 msgr2=0x7f0cc414f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:47.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.825+0000 7f0cc961e640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cc414f420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0cb402cb70 tx=0x7f0cc40b2400 comp rx=0 tx=0).stop 2023-12-12T09:53:47.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.827+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 shutdown_connections 2023-12-12T09:53:47.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.827+0000 7f0cc961e640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0ca406ae70 0x7f0ca406d330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.827+0000 7f0cc961e640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cc40ca700 0x7f0cc40b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.827+0000 7f0cc961e640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cc40cbce0 0x7f0cc40ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.827+0000 7f0cc961e640 1 --2- 172.21.15.72:0/4285139068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb0003970 0x7f0cc414f420 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:47.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.827+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 >> 172.21.15.72:0/4285139068 conn(0x7f0cc409c420 msgr2=0x7f0cc40b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:47.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.828+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 shutdown_connections 2023-12-12T09:53:47.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:47.829+0000 7f0cc961e640 1 -- 172.21.15.72:0/4285139068 wait complete. 2023-12-12T09:53:47.860 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:48.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.038+0000 7f176789b640 1 Processor -- start 2023-12-12T09:53:48.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.038+0000 7f176789b640 1 -- start start 2023-12-12T09:53:48.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.039+0000 7f176789b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f176006c410 0x7f176015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.039+0000 7f176789b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f176015e0e0 0x7f17600cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.039+0000 7f176789b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176006ca40 con 0x7f1760157a60 2023-12-12T09:53:48.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.039+0000 7f176789b640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176015e550 con 0x7f176006c410 2023-12-12T09:53:48.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.039+0000 7f176789b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1760071cd0 con 0x7f176015e0e0 2023-12-12T09:53:48.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.039+0000 7f1765637640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1760157a60 0x7f176006de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59924/0 (socket says 172.21.15.72:59924) 2023-12-12T09:53:48.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.039+0000 7f1765637640 1 -- 172.21.15.72:0/2757374357 learned_addr learned my addr 172.21.15.72:0/2757374357 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:48.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.039+0000 7f1764e36640 1 --2- 172.21.15.72:0/2757374357 >> v2:172.21.15.72:3300/0 conn(0x7f176006c410 0x7f176015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.040+0000 7f1765e38640 1 --2- 172.21.15.72:0/2757374357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f176015e0e0 0x7f17600cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.040+0000 7f1753fff640 1 -- 172.21.15.72:0/2757374357 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500473238 0 0) 0x7f176006ca40 con 0x7f1760157a60 2023-12-12T09:53:48.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.040+0000 7f1753fff640 1 -- 172.21.15.72:0/2757374357 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f174c003520 con 0x7f1760157a60 2023-12-12T09:53:48.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.040+0000 7f1764e36640 1 -- 172.21.15.72:0/2757374357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f176015e0e0 msgr2=0x7f17600cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:48.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.040+0000 7f1764e36640 1 --2- 172.21.15.72:0/2757374357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f176015e0e0 0x7f17600cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.040+0000 7f1764e36640 1 -- 172.21.15.72:0/2757374357 >> v1:172.21.15.72:6789/0 conn(0x7f1760157a60 legacy=0x7f176006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.040+0000 7f1764e36640 1 -- 172.21.15.72:0/2757374357 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1760070f10 con 0x7f176006c410 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.041+0000 7f1764e36640 1 --2- 172.21.15.72:0/2757374357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176006c410 0x7f176015dba0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f175c009d90 tx=0x7f175c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1d0e427eb5dab server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.041+0000 7f1753fff640 1 -- 172.21.15.72:0/2757374357 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c02cbc0 con 0x7f176006c410 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.041+0000 7f1753fff640 1 -- 172.21.15.72:0/2757374357 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c02cd20 con 0x7f176006c410 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.041+0000 7f1753fff640 1 -- 172.21.15.72:0/2757374357 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c034660 con 0x7f176006c410 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.042+0000 7f176789b640 1 -- 172.21.15.72:0/2757374357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176006c410 msgr2=0x7f176015dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.042+0000 7f176789b640 1 --2- 172.21.15.72:0/2757374357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176006c410 0x7f176015dba0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f175c009d90 tx=0x7f175c02be10 comp rx=0 tx=0).stop 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.042+0000 7f176789b640 1 -- 172.21.15.72:0/2757374357 shutdown_connections 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.042+0000 7f176789b640 1 --2- 172.21.15.72:0/2757374357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f176015e0e0 0x7f17600cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.042+0000 7f176789b640 1 --2- 172.21.15.72:0/2757374357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176006c410 0x7f176015dba0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.042+0000 7f176789b640 1 -- 172.21.15.72:0/2757374357 >> 172.21.15.72:0/2757374357 conn(0x7f17600b0a90 msgr2=0x7f17600b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.042+0000 7f176789b640 1 -- 172.21.15.72:0/2757374357 shutdown_connections 2023-12-12T09:53:48.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.042+0000 7f176789b640 1 -- 172.21.15.72:0/2757374357 wait complete. 2023-12-12T09:53:48.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.044+0000 7f176789b640 1 Processor -- start 2023-12-12T09:53:48.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.044+0000 7f176789b640 1 -- start start 2023-12-12T09:53:48.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f176789b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f176006c410 0x7f176014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f176789b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1760157a60 0x7f1760145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f176789b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176015e0e0 0x7f176014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f176789b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176015ca20 con 0x7f176015e0e0 2023-12-12T09:53:48.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f176789b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1760145650 con 0x7f176006c410 2023-12-12T09:53:48.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f176789b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176014f7d0 con 0x7f1760157a60 2023-12-12T09:53:48.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f1765e38640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176015e0e0 0x7f176014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f1765e38640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176015e0e0 0x7f176014fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52132/0 (socket says 172.21.15.72:52132) 2023-12-12T09:53:48.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.045+0000 7f1765e38640 1 -- 172.21.15.72:0/309388653 learned_addr learned my addr 172.21.15.72:0/309388653 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:48.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1764e36640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1760157a60 0x7f1760145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1765e38640 1 -- 172.21.15.72:0/309388653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1760157a60 msgr2=0x7f1760145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1765e38640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1760157a60 0x7f1760145110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1765637640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f176006c410 0x7f176014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1765e38640 1 -- 172.21.15.72:0/309388653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f176006c410 msgr2=0x7f176014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1765e38640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f176006c410 0x7f176014f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1765e38640 1 -- 172.21.15.72:0/309388653 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c009a70 con 0x7f176015e0e0 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1765e38640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176015e0e0 0x7f176014fbb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f175800d5d0 tx=0x7f175800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1751ffb640 1 -- 172.21.15.72:0/309388653 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17580125f0 con 0x7f176015e0e0 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1751ffb640 1 -- 172.21.15.72:0/309388653 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1758012770 con 0x7f176015e0e0 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f1751ffb640 1 -- 172.21.15.72:0/309388653 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758012b20 con 0x7f176015e0e0 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176015c0f0 con 0x7f176015e0e0 2023-12-12T09:53:48.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.046+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176015c6b0 con 0x7f176015e0e0 2023-12-12T09:53:48.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.048+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176015cb90 con 0x7f176015e0e0 2023-12-12T09:53:48.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.048+0000 7f1751ffb640 1 -- 172.21.15.72:0/309388653 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1758012c80 con 0x7f176015e0e0 2023-12-12T09:53:48.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.049+0000 7f1751ffb640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.049+0000 7f1751ffb640 1 -- 172.21.15.72:0/309388653 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1758089cc0 con 0x7f176015e0e0 2023-12-12T09:53:48.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.050+0000 7f1765637640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.050+0000 7f1765637640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f174406ae40 0x7f174406d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f17540011e0 tx=0x7f1754009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.054+0000 7f1751ffb640 1 -- 172.21.15.72:0/309388653 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1758090080 con 0x7f176015e0e0 2023-12-12T09:53:48.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.246+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17600745d0 con 0x7f176015e0e0 2023-12-12T09:53:48.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.249+0000 7f1751ffb640 1 -- 172.21.15.72:0/309388653 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f1758054ba0 con 0x7f176015e0e0 2023-12-12T09:53:48.252 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:48.252 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":37,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:53:47.253690+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:53:45.083566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-12T09:53:46.274070+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f174406ae40 msgr2=0x7f174406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f174406ae40 0x7f174406d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f17540011e0 tx=0x7f1754009040 comp rx=0 tx=0).stop 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176015e0e0 msgr2=0x7f176014fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176015e0e0 0x7f176014fbb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f175800d5d0 tx=0x7f175800da90 comp rx=0 tx=0).stop 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 shutdown_connections 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176015e0e0 0x7f176014fbb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1760157a60 0x7f1760145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 --2- 172.21.15.72:0/309388653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f176006c410 0x7f176014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 >> 172.21.15.72:0/309388653 conn(0x7f17600b0a90 msgr2=0x7f176015a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:48.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 shutdown_connections 2023-12-12T09:53:48.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.252+0000 7f176789b640 1 -- 172.21.15.72:0/309388653 wait complete. 2023-12-12T09:53:48.273 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:48.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.473+0000 7f66ec0d0640 1 Processor -- start 2023-12-12T09:53:48.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.473+0000 7f66ec0d0640 1 -- start start 2023-12-12T09:53:48.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.474+0000 7f66ec0d0640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f66e406c410 0x7f66e415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.474+0000 7f66ec0d0640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e406ca40 con 0x7f66e415e0e0 2023-12-12T09:53:48.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.475+0000 7f66ec0d0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e415e510 con 0x7f66e4157a60 2023-12-12T09:53:48.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.475+0000 7f66ec0d0640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e4071cd0 con 0x7f66e406c410 2023-12-12T09:53:48.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.475+0000 7f66ea66d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f66e415e0e0 0x7f66e40cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59938/0 (socket says 172.21.15.72:59938) 2023-12-12T09:53:48.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.475+0000 7f66ea66d640 1 -- 172.21.15.72:0/4171202474 learned_addr learned my addr 172.21.15.72:0/4171202474 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:48.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.475+0000 7f66e966b640 1 --2- 172.21.15.72:0/4171202474 >> v2:172.21.15.72:3300/0 conn(0x7f66e406c410 0x7f66e415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.475+0000 7f66e8e6a640 1 -- 172.21.15.72:0/4171202474 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096257386 0 0) 0x7f66e406ca40 con 0x7f66e415e0e0 2023-12-12T09:53:48.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.475+0000 7f66e8e6a640 1 -- 172.21.15.72:0/4171202474 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66d0005730 con 0x7f66e415e0e0 2023-12-12T09:53:48.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.476+0000 7f66e966b640 1 -- 172.21.15.72:0/4171202474 >> v1:172.21.15.113:6789/0 conn(0x7f66e4157a60 legacy=0x7f66e406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.476+0000 7f66e966b640 1 -- 172.21.15.72:0/4171202474 >> v1:172.21.15.72:6789/0 conn(0x7f66e415e0e0 legacy=0x7f66e40cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.476+0000 7f66e966b640 1 -- 172.21.15.72:0/4171202474 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e4070f10 con 0x7f66e406c410 2023-12-12T09:53:48.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.477+0000 7f66e966b640 1 --2- 172.21.15.72:0/4171202474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e406c410 0x7f66e415dba0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f66d40097a0 tx=0x7f66d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4272167f994e5a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.477+0000 7f66e8e6a640 1 -- 172.21.15.72:0/4171202474 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4031830 con 0x7f66e406c410 2023-12-12T09:53:48.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.477+0000 7f66e8e6a640 1 -- 172.21.15.72:0/4171202474 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d4031990 con 0x7f66e406c410 2023-12-12T09:53:48.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.477+0000 7f66e8e6a640 1 -- 172.21.15.72:0/4171202474 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4031d20 con 0x7f66e406c410 2023-12-12T09:53:48.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.477+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4171202474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e406c410 msgr2=0x7f66e415dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.477+0000 7f66ec0d0640 1 --2- 172.21.15.72:0/4171202474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e406c410 0x7f66e415dba0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f66d40097a0 tx=0x7f66d402b8d0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.478+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4171202474 shutdown_connections 2023-12-12T09:53:48.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.478+0000 7f66ec0d0640 1 --2- 172.21.15.72:0/4171202474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e406c410 0x7f66e415dba0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.478+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4171202474 >> 172.21.15.72:0/4171202474 conn(0x7f66e40b0a90 msgr2=0x7f66e40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:48.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.478+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4171202474 shutdown_connections 2023-12-12T09:53:48.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.478+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4171202474 wait complete. 2023-12-12T09:53:48.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.479+0000 7f66ec0d0640 1 Processor -- start 2023-12-12T09:53:48.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.480+0000 7f66ec0d0640 1 -- start start 2023-12-12T09:53:48.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.480+0000 7f66ec0d0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66e406c410 0x7f66e4155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.480+0000 7f66ec0d0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e4157a60 0x7f66e40b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66ec0d0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e415e0e0 0x7f66e40ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66ec0d0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4156040 con 0x7f66e4157a60 2023-12-12T09:53:48.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66ec0d0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40b08e0 con 0x7f66e415e0e0 2023-12-12T09:53:48.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66ec0d0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4148bd0 con 0x7f66e406c410 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e966b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e4157a60 0x7f66e40b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e966b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e4157a60 0x7f66e40b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52140/0 (socket says 172.21.15.72:52140) 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e966b640 1 -- 172.21.15.72:0/4286014793 learned_addr learned my addr 172.21.15.72:0/4286014793 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e9e6c640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66e406c410 0x7f66e4155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e966b640 1 -- 172.21.15.72:0/4286014793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66e406c410 msgr2=0x7f66e4155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e966b640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66e406c410 0x7f66e4155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66ea66d640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e415e0e0 0x7f66e40ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e966b640 1 -- 172.21.15.72:0/4286014793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e415e0e0 msgr2=0x7f66e40ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e966b640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e415e0e0 0x7f66e40ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.481+0000 7f66e966b640 1 -- 172.21.15.72:0/4286014793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d4009400 con 0x7f66e4157a60 2023-12-12T09:53:48.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.482+0000 7f66e966b640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e4157a60 0x7f66e40b03a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f66d4031e80 tx=0x7f66d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.482+0000 7f66da7fc640 1 -- 172.21.15.72:0/4286014793 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d403a6f0 con 0x7f66e4157a60 2023-12-12T09:53:48.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.482+0000 7f66da7fc640 1 -- 172.21.15.72:0/4286014793 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d403a870 con 0x7f66e4157a60 2023-12-12T09:53:48.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.482+0000 7f66da7fc640 1 -- 172.21.15.72:0/4286014793 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d403ac20 con 0x7f66e4157a60 2023-12-12T09:53:48.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.482+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e40ae6c0 con 0x7f66e4157a60 2023-12-12T09:53:48.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.482+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e4156ee0 con 0x7f66e4157a60 2023-12-12T09:53:48.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.483+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e40745d0 con 0x7f66e4157a60 2023-12-12T09:53:48.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.484+0000 7f66da7fc640 1 -- 172.21.15.72:0/4286014793 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66d4030380 con 0x7f66e4157a60 2023-12-12T09:53:48.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.485+0000 7f66da7fc640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f66c806ae40 0x7f66c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.485+0000 7f66da7fc640 1 -- 172.21.15.72:0/4286014793 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f66d40ab070 con 0x7f66e4157a60 2023-12-12T09:53:48.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.485+0000 7f66e9e6c640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f66c806ae40 0x7f66c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.486+0000 7f66e9e6c640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f66c806ae40 0x7f66c806d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000f60 tx=0x7f66e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.490+0000 7f66da7fc640 1 -- 172.21.15.72:0/4286014793 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66d4075ff0 con 0x7f66e4157a60 2023-12-12T09:53:48.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.680+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66e41530b0 con 0x7f66e4157a60 2023-12-12T09:53:48.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.683+0000 7f66da7fc640 1 -- 172.21.15.72:0/4286014793 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f66d4079610 con 0x7f66e4157a60 2023-12-12T09:53:48.685 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:48.686 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":37,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:53:47.253690+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:53:45.083566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-12T09:53:46.274070+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f66c806ae40 msgr2=0x7f66c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f66c806ae40 0x7f66c806d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000f60 tx=0x7f66e0009040 comp rx=0 tx=0).stop 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e4157a60 msgr2=0x7f66e40b03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e4157a60 0x7f66e40b03a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f66d4031e80 tx=0x7f66d40046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 shutdown_connections 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f66c806ae40 0x7f66c806d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e415e0e0 0x7f66e40ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66e4157a60 0x7f66e40b03a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 --2- 172.21.15.72:0/4286014793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66e406c410 0x7f66e4155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 >> 172.21.15.72:0/4286014793 conn(0x7f66e40b0a90 msgr2=0x7f66e40b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:48.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 shutdown_connections 2023-12-12T09:53:48.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.686+0000 7f66ec0d0640 1 -- 172.21.15.72:0/4286014793 wait complete. 2023-12-12T09:53:48.707 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-12T09:53:48.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.915+0000 7f00cf4e7640 1 Processor -- start 2023-12-12T09:53:48.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.915+0000 7f00cf4e7640 1 -- start start 2023-12-12T09:53:48.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.916+0000 7f00cf4e7640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f00c80cc420 0x7f00c8071d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.916+0000 7f00cf4e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00c8072250 0x7f00c80b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.916+0000 7f00cf4e7640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c8159d70 con 0x7f00c8157350 2023-12-12T09:53:48.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.916+0000 7f00cf4e7640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c80726c0 con 0x7f00c80cc420 2023-12-12T09:53:48.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.917+0000 7f00cf4e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c815a030 con 0x7f00c8072250 2023-12-12T09:53:48.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.917+0000 7f00cca82640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f00c80cc420 0x7f00c8071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.917+0000 7f00cd283640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f00c8157350 0x7f00c815dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59944/0 (socket says 172.21.15.72:59944) 2023-12-12T09:53:48.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.917+0000 7f00cd283640 1 -- 172.21.15.72:0/2519441980 learned_addr learned my addr 172.21.15.72:0/2519441980 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:48.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.917+0000 7f00cca82640 1 -- 172.21.15.72:0/2519441980 >> v2:172.21.15.113:3300/0 conn(0x7f00c8072250 msgr2=0x7f00c80b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:48.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.917+0000 7f00cca82640 1 --2- 172.21.15.72:0/2519441980 >> v2:172.21.15.113:3300/0 conn(0x7f00c8072250 0x7f00c80b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.917+0000 7f00cca82640 1 -- 172.21.15.72:0/2519441980 >> v1:172.21.15.72:6789/0 conn(0x7f00c8157350 legacy=0x7f00c815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.917+0000 7f00cca82640 1 -- 172.21.15.72:0/2519441980 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c815a310 con 0x7f00c80cc420 2023-12-12T09:53:48.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.918+0000 7f00cca82640 1 --2- 172.21.15.72:0/2519441980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c80cc420 0x7f00c8071d10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f00c806bdb0 tx=0x7f00b802dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0351e287dcbe115 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00bffff640 1 -- 172.21.15.72:0/2519441980 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b802f8a0 con 0x7f00c80cc420 2023-12-12T09:53:48.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00bffff640 1 -- 172.21.15.72:0/2519441980 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b802fa00 con 0x7f00c80cc420 2023-12-12T09:53:48.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00bffff640 1 -- 172.21.15.72:0/2519441980 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b802fd90 con 0x7f00c80cc420 2023-12-12T09:53:48.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00cf4e7640 1 -- 172.21.15.72:0/2519441980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c80cc420 msgr2=0x7f00c8071d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/2519441980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c80cc420 0x7f00c8071d10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f00c806bdb0 tx=0x7f00b802dae0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00cf4e7640 1 -- 172.21.15.72:0/2519441980 shutdown_connections 2023-12-12T09:53:48.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/2519441980 >> v2:172.21.15.113:3300/0 conn(0x7f00c8072250 0x7f00c80b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/2519441980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c80cc420 0x7f00c8071d10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.919+0000 7f00cf4e7640 1 -- 172.21.15.72:0/2519441980 >> 172.21.15.72:0/2519441980 conn(0x7f00c80cb850 msgr2=0x7f00c80cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:48.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.920+0000 7f00cf4e7640 1 -- 172.21.15.72:0/2519441980 shutdown_connections 2023-12-12T09:53:48.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.920+0000 7f00cf4e7640 1 -- 172.21.15.72:0/2519441980 wait complete. 2023-12-12T09:53:48.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.921+0000 7f00cf4e7640 1 Processor -- start 2023-12-12T09:53:48.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.921+0000 7f00cf4e7640 1 -- start start 2023-12-12T09:53:48.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.922+0000 7f00cf4e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c8072250 0x7f00c8145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.922+0000 7f00cf4e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c80cc420 0x7f00c8155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.922+0000 7f00cf4e7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c8157350 0x7f00c8156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.922+0000 7f00cf4e7640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c8146480 con 0x7f00c8157350 2023-12-12T09:53:48.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.922+0000 7f00cf4e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c80af740 con 0x7f00c80cc420 2023-12-12T09:53:48.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.922+0000 7f00cf4e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c80afa20 con 0x7f00c8072250 2023-12-12T09:53:48.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c8157350 0x7f00c8156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c8157350 0x7f00c8156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52162/0 (socket says 172.21.15.72:52162) 2023-12-12T09:53:48.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 -- 172.21.15.72:0/1011718102 learned_addr learned my addr 172.21.15.72:0/1011718102 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cd283640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c8072250 0x7f00c8145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cca82640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c80cc420 0x7f00c8155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 -- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c8072250 msgr2=0x7f00c8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c8072250 0x7f00c8145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 -- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c80cc420 msgr2=0x7f00c8155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c80cc420 0x7f00c8155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 -- 172.21.15.72:0/1011718102 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b8009400 con 0x7f00c8157350 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cd283640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c8072250 0x7f00c8145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cca82640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c80cc420 0x7f00c8155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.923+0000 7f00cda84640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c8157350 0x7f00c8156410 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f00c000b970 tx=0x7f00c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.924+0000 7f00bdffb640 1 -- 172.21.15.72:0/1011718102 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c000cb40 con 0x7f00c8157350 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.924+0000 7f00bdffb640 1 -- 172.21.15.72:0/1011718102 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c000ccc0 con 0x7f00c8157350 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.924+0000 7f00bdffb640 1 -- 172.21.15.72:0/1011718102 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0014680 con 0x7f00c8157350 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.924+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c80afd00 con 0x7f00c8157350 2023-12-12T09:53:48.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.924+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c80ae980 con 0x7f00c8157350 2023-12-12T09:53:48.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.925+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c80678f0 con 0x7f00c8157350 2023-12-12T09:53:48.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.926+0000 7f00bdffb640 1 -- 172.21.15.72:0/1011718102 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f00c001d3b0 con 0x7f00c8157350 2023-12-12T09:53:48.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.927+0000 7f00bdffb640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f00ac06ae40 0x7f00ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.927+0000 7f00bdffb640 1 -- 172.21.15.72:0/1011718102 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f00c0089720 con 0x7f00c8157350 2023-12-12T09:53:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.927+0000 7f00cd283640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f00ac06ae40 0x7f00ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.928+0000 7f00cd283640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f00ac06ae40 0x7f00ac06d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005880 tx=0x7f00c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:48.931+0000 7f00bdffb640 1 -- 172.21.15.72:0/1011718102 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00c0054670 con 0x7f00c8157350 2023-12-12T09:53:49.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.158+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f00c80745d0 con 0x7f00c8157350 2023-12-12T09:53:49.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.844+0000 7f00bdffb640 1 -- 172.21.15.72:0/1011718102 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f00c0057c90 con 0x7f00c8157350 2023-12-12T09:53:49.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f00ac06ae40 msgr2=0x7f00ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:49.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f00ac06ae40 0x7f00ac06d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005880 tx=0x7f00c4009040 comp rx=0 tx=0).stop 2023-12-12T09:53:49.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c8157350 msgr2=0x7f00c8156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:49.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c8157350 0x7f00c8156410 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f00c000b970 tx=0x7f00c000be30 comp rx=0 tx=0).stop 2023-12-12T09:53:49.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 shutdown_connections 2023-12-12T09:53:49.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f00ac06ae40 0x7f00ac06d300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:49.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00c8157350 0x7f00c8156410 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:49.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c80cc420 0x7f00c8155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:49.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 --2- 172.21.15.72:0/1011718102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c8072250 0x7f00c8145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:49.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 >> 172.21.15.72:0/1011718102 conn(0x7f00c80cb850 msgr2=0x7f00c80ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:49.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 shutdown_connections 2023-12-12T09:53:49.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:49.848+0000 7f00cf4e7640 1 -- 172.21.15.72:0/1011718102 wait complete. 2023-12-12T09:53:49.871 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-12T09:53:49.871 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-12T09:53:50.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.084+0000 7fe4f3b9e640 1 Processor -- start 2023-12-12T09:53:50.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.084+0000 7fe4f3b9e640 1 -- start start 2023-12-12T09:53:50.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f3b9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4ec141960 0x7fe4ec166100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:50.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f3b9e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ec15b780 con 0x7fe4ec06d770 2023-12-12T09:53:50.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f3b9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ec0b2510 con 0x7fe4ec06db50 2023-12-12T09:53:50.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f3b9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec15b9d0 con 0x7fe4ec141960 2023-12-12T09:53:50.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f193a640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fe4ec06d770 0x7fe4ec0b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59946/0 (socket says 172.21.15.72:59946) 2023-12-12T09:53:50.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f193a640 1 -- 172.21.15.72:0/3168507768 learned_addr learned my addr 172.21.15.72:0/3168507768 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:50.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575066230 0 0) 0x7fe4ec15b780 con 0x7fe4ec06d770 2023-12-12T09:53:50.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d8003520 con 0x7fe4ec06d770 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.086+0000 7fe4f213b640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4ec141960 0x7fe4ec166100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3215565008 0 0) 0x7fe4d8003520 con 0x7fe4ec06d770 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4ec141960 msgr2=0x7fe4ec166100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4ec141960 0x7fe4ec166100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 >> v1:172.21.15.113:6789/0 conn(0x7fe4ec06db50 legacy=0x7fe4ec141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec15bcb0 con 0x7fe4ec06d770 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fe4e8001670 con 0x7fe4ec06d770 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 >> v1:172.21.15.72:6789/0 conn(0x7fe4ec06d770 legacy=0x7fe4ec0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4d80039a0 0x7fe4d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4d80047c0 0x7fe4d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4ec141960 0x7fe4d8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec15b9d0 con 0x7fe4d80039a0 2023-12-12T09:53:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d800bb40 con 0x7fe4d80047c0 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d800bd00 con 0x7fe4ec141960 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f193a640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4d80039a0 0x7fe4d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f193a640 1 -- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4ec141960 msgr2=0x7fe4d8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f193a640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4ec141960 0x7fe4d8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.087+0000 7fe4f193a640 1 -- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4d80047c0 msgr2=0x7fe4d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.088+0000 7fe4f1139640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4d80047c0 0x7fe4d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.088+0000 7fe4f193a640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4d80047c0 0x7fe4d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.088+0000 7fe4f193a640 1 -- 172.21.15.72:0/3168507768 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d8003520 con 0x7fe4d80039a0 2023-12-12T09:53:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.088+0000 7fe4f1139640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4d80047c0 0x7fe4d8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:50.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.089+0000 7fe4f193a640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4d80039a0 0x7fe4d8003dc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c0e0 tx=0x7fe4d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a20a3c8a154c80 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:50.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.089+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8005ad0 con 0x7fe4d80039a0 2023-12-12T09:53:50.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.089+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e8005c30 con 0x7fe4d80039a0 2023-12-12T09:53:50.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.089+0000 7fe4f0938640 1 -- 172.21.15.72:0/3168507768 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e802c810 con 0x7fe4d80039a0 2023-12-12T09:53:50.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.089+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/3168507768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4d80039a0 msgr2=0x7fe4d8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:50.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.089+0000 7fe4f3b9e640 1 --2- 172.21.15.72:0/3168507768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4d80039a0 0x7fe4d8003dc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800c0e0 tx=0x7fe4d80057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.089+0000 7fe4f213b640 1 -- 172.21.15.72:0/3168507768 reap_dead start 2023-12-12T09:53:50.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.090+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/3168507768 shutdown_connections 2023-12-12T09:53:50.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.090+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/3168507768 >> 172.21.15.72:0/3168507768 conn(0x7fe4ec0cb850 msgr2=0x7fe4ec0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:50.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.090+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/3168507768 shutdown_connections 2023-12-12T09:53:50.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.090+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/3168507768 wait complete. 2023-12-12T09:53:50.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.091+0000 7fe4f3b9e640 1 Processor -- start 2023-12-12T09:53:50.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.091+0000 7fe4f3b9e640 1 -- start start 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.092+0000 7fe4f3b9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4d80039a0 0x7fe4ec0ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.092+0000 7fe4f3b9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4ec15a200 0x7fe4ec1e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.092+0000 7fe4f3b9e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4ec159c40 0x7fe4ec1e7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.092+0000 7fe4f3b9e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec1e5ea0 con 0x7fe4ec159c40 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.092+0000 7fe4f3b9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec15d9b0 con 0x7fe4ec15a200 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.092+0000 7fe4f3b9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec15dc90 con 0x7fe4d80039a0 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4ec159c40 0x7fe4ec1e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4ec159c40 0x7fe4ec1e7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52178/0 (socket says 172.21.15.72:52178) 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 -- 172.21.15.72:0/1776871859 learned_addr learned my addr 172.21.15.72:0/1776871859 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:50.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 -- 172.21.15.72:0/1776871859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4d80039a0 msgr2=0x7fe4ec0ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4d80039a0 0x7fe4ec0ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 -- 172.21.15.72:0/1776871859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4ec15a200 msgr2=0x7fe4ec1e6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4ec15a200 0x7fe4ec1e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 -- 172.21.15.72:0/1776871859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec15df70 con 0x7fe4ec159c40 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.093+0000 7fe4f213b640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4ec159c40 0x7fe4ec1e7500 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0b42d0 tx=0x7fe4e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.094+0000 7fe4de7fc640 1 -- 172.21.15.72:0/1776871859 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4018070 con 0x7fe4ec159c40 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.094+0000 7fe4de7fc640 1 -- 172.21.15.72:0/1776871859 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e400eb90 con 0x7fe4ec159c40 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.094+0000 7fe4de7fc640 1 -- 172.21.15.72:0/1776871859 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4003990 con 0x7fe4ec159c40 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.094+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ec1e6180 con 0x7fe4ec159c40 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.094+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec1e4bb0 con 0x7fe4ec159c40 2023-12-12T09:53:50.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.095+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec0745d0 con 0x7fe4ec159c40 2023-12-12T09:53:50.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.096+0000 7fe4de7fc640 1 -- 172.21.15.72:0/1776871859 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4e4014420 con 0x7fe4ec159c40 2023-12-12T09:53:50.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.097+0000 7fe4de7fc640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4d006ae70 0x7fe4d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:50.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.097+0000 7fe4de7fc640 1 -- 172.21.15.72:0/1776871859 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe4e4089580 con 0x7fe4ec159c40 2023-12-12T09:53:50.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.097+0000 7fe4f193a640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4d006ae70 0x7fe4d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:50.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.098+0000 7fe4f193a640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4d006ae70 0x7fe4d006d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8034000 tx=0x7fe4e8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:50.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.101+0000 7fe4de7fc640 1 -- 172.21.15.72:0/1776871859 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4e40544e0 con 0x7fe4ec159c40 2023-12-12T09:53:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.328+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe4ec071d30 con 0x7fe4ec159c40 2023-12-12T09:53:50.753 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-12-12T09:53:50.751+0000 7ff6fe0f5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-12T09:53:50.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.857+0000 7fe4de7fc640 1 -- 172.21.15.72:0/1776871859 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4e40097a0 con 0x7fe4ec159c40 2023-12-12T09:53:50.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4d006ae70 msgr2=0x7fe4d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4d006ae70 0x7fe4d006d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8034000 tx=0x7fe4e8007000 comp rx=0 tx=0).stop 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4ec159c40 msgr2=0x7fe4ec1e7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4ec159c40 0x7fe4ec1e7500 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0b42d0 tx=0x7fe4e400e4b0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 shutdown_connections 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4d006ae70 0x7fe4d006d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4ec159c40 0x7fe4ec1e7500 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4ec15a200 0x7fe4ec1e6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 --2- 172.21.15.72:0/1776871859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4d80039a0 0x7fe4ec0ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.861+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 >> 172.21.15.72:0/1776871859 conn(0x7fe4ec0cb850 msgr2=0x7fe4ec15b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.862+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 shutdown_connections 2023-12-12T09:53:50.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:50.862+0000 7fe4f3b9e640 1 -- 172.21.15.72:0/1776871859 wait complete. 2023-12-12T09:53:50.884 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-12T09:53:50.885 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:53:51.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.084+0000 7f77884da640 1 Processor -- start 2023-12-12T09:53:51.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.084+0000 7f77884da640 1 -- start start 2023-12-12T09:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f77884da640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7780143e00 0x7f77800b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f77884da640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7780072180 0x7f77800b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f77884da640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77800b3bb0 con 0x7f778006c410 2023-12-12T09:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f77884da640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77801441e0 con 0x7f7780143e00 2023-12-12T09:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f77884da640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7780072680 con 0x7f7780072180 2023-12-12T09:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f7786276640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7780143e00 0x7f77800b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f7786276640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7780143e00 0x7f77800b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52188/0 (socket says 172.21.15.72:52188) 2023-12-12T09:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f7786276640 1 -- 172.21.15.72:0/1249503697 learned_addr learned my addr 172.21.15.72:0/1249503697 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:51.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f7785a75640 1 --2- 172.21.15.72:0/1249503697 >> v2:172.21.15.113:3300/0 conn(0x7f7780072180 0x7f77800b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f7786276640 1 -- 172.21.15.72:0/1249503697 >> v2:172.21.15.113:3300/0 conn(0x7f7780072180 msgr2=0x7f77800b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:51.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.086+0000 7f7786276640 1 --2- 172.21.15.72:0/1249503697 >> v2:172.21.15.113:3300/0 conn(0x7f7780072180 0x7f77800b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.087+0000 7f7786276640 1 -- 172.21.15.72:0/1249503697 >> v1:172.21.15.113:6789/0 conn(0x7f778006c410 legacy=0x7f7780070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:51.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.087+0000 7f7786276640 1 -- 172.21.15.72:0/1249503697 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77800b1e30 con 0x7f7780143e00 2023-12-12T09:53:51.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.088+0000 7f7786276640 1 --2- 172.21.15.72:0/1249503697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7780143e00 0x7f77800b4040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7780067da0 tx=0x7f777c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fc35945452e411 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:51.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.088+0000 7f7785274640 1 -- 172.21.15.72:0/1249503697 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c031870 con 0x7f7780143e00 2023-12-12T09:53:51.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.088+0000 7f7785274640 1 -- 172.21.15.72:0/1249503697 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c0319d0 con 0x7f7780143e00 2023-12-12T09:53:51.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.088+0000 7f7785274640 1 -- 172.21.15.72:0/1249503697 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c031d60 con 0x7f7780143e00 2023-12-12T09:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.089+0000 7f77884da640 1 -- 172.21.15.72:0/1249503697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7780143e00 msgr2=0x7f77800b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.089+0000 7f77884da640 1 --2- 172.21.15.72:0/1249503697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7780143e00 0x7f77800b4040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7780067da0 tx=0x7f777c02b910 comp rx=0 tx=0).stop 2023-12-12T09:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.089+0000 7f77884da640 1 -- 172.21.15.72:0/1249503697 shutdown_connections 2023-12-12T09:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.089+0000 7f77884da640 1 --2- 172.21.15.72:0/1249503697 >> v2:172.21.15.113:3300/0 conn(0x7f7780072180 0x7f77800b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.089+0000 7f77884da640 1 --2- 172.21.15.72:0/1249503697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7780143e00 0x7f77800b4040 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.089+0000 7f77884da640 1 -- 172.21.15.72:0/1249503697 >> 172.21.15.72:0/1249503697 conn(0x7f77800b0a90 msgr2=0x7f77800b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.089+0000 7f77884da640 1 -- 172.21.15.72:0/1249503697 shutdown_connections 2023-12-12T09:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.089+0000 7f77884da640 1 -- 172.21.15.72:0/1249503697 wait complete. 2023-12-12T09:53:51.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.091+0000 7f77884da640 1 Processor -- start 2023-12-12T09:53:51.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.091+0000 7f77884da640 1 -- start start 2023-12-12T09:53:51.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f77884da640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f778006c410 0x7f77800cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f77884da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780072180 0x7f7780140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f77884da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7780143e00 0x7f77801f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f77884da640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7780141b10 con 0x7f778006c410 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f77884da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7780140c80 con 0x7f7780072180 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f77884da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7780140f00 con 0x7f7780143e00 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f7786276640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f778006c410 0x7f77800cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f7786276640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f778006c410 0x7f77800cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52194/0 (socket says 172.21.15.72:52194) 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.092+0000 7f7786276640 1 -- 172.21.15.72:0/4245780614 learned_addr learned my addr 172.21.15.72:0/4245780614 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7785a75640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780072180 0x7f7780140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7786a77640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7780143e00 0x7f77801f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7786276640 1 -- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7780143e00 msgr2=0x7f77801f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7786276640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7780143e00 0x7f77801f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7786276640 1 -- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780072180 msgr2=0x7f7780140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7786276640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780072180 0x7f7780140740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7786276640 1 -- 172.21.15.72:0/4245780614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c0094d0 con 0x7f778006c410 2023-12-12T09:53:51.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7785a75640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780072180 0x7f7780140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7786a77640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7780143e00 0x7f77801f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7786276640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f778006c410 0x7f77800cde30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f777c000c00 tx=0x7f777c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7776ffd640 1 -- 172.21.15.72:0/4245780614 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c03abe0 con 0x7f778006c410 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7776ffd640 1 -- 172.21.15.72:0/4245780614 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c03ad40 con 0x7f778006c410 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77801411e0 con 0x7f778006c410 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.093+0000 7f7776ffd640 1 -- 172.21.15.72:0/4245780614 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c044610 con 0x7f778006c410 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.094+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77801417a0 con 0x7f778006c410 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.095+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77800cb5e0 con 0x7f778006c410 2023-12-12T09:53:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.096+0000 7f7776ffd640 1 -- 172.21.15.72:0/4245780614 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f777c04d450 con 0x7f778006c410 2023-12-12T09:53:51.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.096+0000 7f7776ffd640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f776406ae40 0x7f776406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.097+0000 7f7776ffd640 1 -- 172.21.15.72:0/4245780614 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f777c02d040 con 0x7f778006c410 2023-12-12T09:53:51.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.097+0000 7f7785a75640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f776406ae40 0x7f776406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.098+0000 7f7785a75640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f776406ae40 0x7f776406d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7780138730 tx=0x7f7770008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:51.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.101+0000 7f7776ffd640 1 -- 172.21.15.72:0/4245780614 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f777c076c80 con 0x7f778006c410 2023-12-12T09:53:51.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.337+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77800745d0 con 0x7f778006c410 2023-12-12T09:53:51.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.338+0000 7f7776ffd640 1 -- 172.21.15.72:0/4245780614 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f777c0abe60 con 0x7f778006c410 2023-12-12T09:53:51.340 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:51.340 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:53:51.340 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.340+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f776406ae40 msgr2=0x7f776406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.340+0000 7f77884da640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f776406ae40 0x7f776406d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7780138730 tx=0x7f7770008040 comp rx=0 tx=0).stop 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.340+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f778006c410 msgr2=0x7f77800cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.340+0000 7f77884da640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f778006c410 0x7f77800cde30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f777c000c00 tx=0x7f777c0046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.340+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 shutdown_connections 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.341+0000 7f77884da640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f776406ae40 0x7f776406d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.341+0000 7f77884da640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7780143e00 0x7f77801f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.341+0000 7f77884da640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780072180 0x7f7780140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.341+0000 7f77884da640 1 --2- 172.21.15.72:0/4245780614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f778006c410 0x7f77800cde30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.341+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 >> 172.21.15.72:0/4245780614 conn(0x7f77800b0a90 msgr2=0x7f77800af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:51.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.341+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 shutdown_connections 2023-12-12T09:53:51.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.341+0000 7f77884da640 1 -- 172.21.15.72:0/4245780614 wait complete. 2023-12-12T09:53:51.361 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:53:51.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.562+0000 7f5b31cff640 1 Processor -- start 2023-12-12T09:53:51.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.562+0000 7f5b31cff640 1 -- start start 2023-12-12T09:53:51.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.563+0000 7f5b31cff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5b2c0713b0 0x7f5b2c0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.563+0000 7f5b31cff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b2c070750 0x7f5b2c0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.563+0000 7f5b31cff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c070070 con 0x7f5b2c159fd0 2023-12-12T09:53:51.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.563+0000 7f5b31cff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c0705f0 con 0x7f5b2c0713b0 2023-12-12T09:53:51.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.564+0000 7f5b31cff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c070310 con 0x7f5b2c070750 2023-12-12T09:53:51.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.564+0000 7f5b2bfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5b2c0713b0 0x7f5b2c0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.564+0000 7f5b30cfd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f5b2c159fd0 0x7f5b2c070ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59960/0 (socket says 172.21.15.72:59960) 2023-12-12T09:53:51.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.564+0000 7f5b30cfd640 1 -- 172.21.15.72:0/2535402365 learned_addr learned my addr 172.21.15.72:0/2535402365 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:51.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.564+0000 7f5b2bfff640 1 -- 172.21.15.72:0/2535402365 >> v2:172.21.15.113:3300/0 conn(0x7f5b2c070750 msgr2=0x7f5b2c0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:51.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.564+0000 7f5b2bfff640 1 --2- 172.21.15.72:0/2535402365 >> v2:172.21.15.113:3300/0 conn(0x7f5b2c070750 0x7f5b2c0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.564+0000 7f5b2bfff640 1 -- 172.21.15.72:0/2535402365 >> v1:172.21.15.72:6789/0 conn(0x7f5b2c159fd0 legacy=0x7f5b2c070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.564+0000 7f5b2bfff640 1 -- 172.21.15.72:0/2535402365 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0b1d00 con 0x7f5b2c0713b0 2023-12-12T09:53:51.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.566+0000 7f5b2bfff640 1 --2- 172.21.15.72:0/2535402365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c0ae1d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0cfbb0 tx=0x7f5b1802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b56b8eb27b40f66 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:51.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.566+0000 7f5b2b7fe640 1 -- 172.21.15.72:0/2535402365 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18030610 con 0x7f5b2c0713b0 2023-12-12T09:53:51.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.566+0000 7f5b2b7fe640 1 -- 172.21.15.72:0/2535402365 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b18030770 con 0x7f5b2c0713b0 2023-12-12T09:53:51.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.566+0000 7f5b2b7fe640 1 -- 172.21.15.72:0/2535402365 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18030b00 con 0x7f5b2c0713b0 2023-12-12T09:53:51.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.567+0000 7f5b31cff640 1 -- 172.21.15.72:0/2535402365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 msgr2=0x7f5b2c0ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.567+0000 7f5b31cff640 1 --2- 172.21.15.72:0/2535402365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c0ae1d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0cfbb0 tx=0x7f5b1802b960 comp rx=0 tx=0).stop 2023-12-12T09:53:51.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.567+0000 7f5b31cff640 1 -- 172.21.15.72:0/2535402365 shutdown_connections 2023-12-12T09:53:51.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.567+0000 7f5b31cff640 1 --2- 172.21.15.72:0/2535402365 >> v2:172.21.15.113:3300/0 conn(0x7f5b2c070750 0x7f5b2c0b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.567+0000 7f5b31cff640 1 --2- 172.21.15.72:0/2535402365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c0ae1d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.567+0000 7f5b31cff640 1 -- 172.21.15.72:0/2535402365 >> 172.21.15.72:0/2535402365 conn(0x7f5b2c0b0860 msgr2=0x7f5b2c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:51.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.567+0000 7f5b31cff640 1 -- 172.21.15.72:0/2535402365 shutdown_connections 2023-12-12T09:53:51.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.567+0000 7f5b31cff640 1 -- 172.21.15.72:0/2535402365 wait complete. 2023-12-12T09:53:51.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.569+0000 7f5b31cff640 1 Processor -- start 2023-12-12T09:53:51.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.569+0000 7f5b31cff640 1 -- start start 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b31cff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c070750 0x7f5b2c143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b31cff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b31cff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2c159fd0 0x7f5b2c152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b31cff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c150810 con 0x7f5b2c0713b0 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b31cff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c150af0 con 0x7f5b2c070750 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b31cff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c150dd0 con 0x7f5b2c159fd0 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52218/0 (socket says 172.21.15.72:52218) 2023-12-12T09:53:51.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.570+0000 7f5b2bfff640 1 -- 172.21.15.72:0/250474424 learned_addr learned my addr 172.21.15.72:0/250474424 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b30cfd640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c070750 0x7f5b2c143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b2bfff640 1 -- 172.21.15.72:0/250474424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2c159fd0 msgr2=0x7f5b2c152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b2bfff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2c159fd0 0x7f5b2c152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b2bfff640 1 -- 172.21.15.72:0/250474424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c070750 msgr2=0x7f5b2c143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b2bfff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c070750 0x7f5b2c143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b2bfff640 1 -- 172.21.15.72:0/250474424 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18009400 con 0x7f5b2c0713b0 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b30cfd640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c070750 0x7f5b2c143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b2bfff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c151de0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009890 tx=0x7f5b18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b297fa640 1 -- 172.21.15.72:0/250474424 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1803e040 con 0x7f5b2c0713b0 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b297fa640 1 -- 172.21.15.72:0/250474424 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b18039c90 con 0x7f5b2c0713b0 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b297fa640 1 -- 172.21.15.72:0/250474424 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b180443f0 con 0x7f5b2c0713b0 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.571+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c1510b0 con 0x7f5b2c0713b0 2023-12-12T09:53:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.572+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c14fae0 con 0x7f5b2c0713b0 2023-12-12T09:53:51.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.573+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c0ad710 con 0x7f5b2c0713b0 2023-12-12T09:53:51.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.574+0000 7f5b297fa640 1 -- 172.21.15.72:0/250474424 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5b1804d3b0 con 0x7f5b2c0713b0 2023-12-12T09:53:51.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.574+0000 7f5b297fa640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5b1006ae40 0x7f5b1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:51.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.574+0000 7f5b297fa640 1 -- 172.21.15.72:0/250474424 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5b180abe80 con 0x7f5b2c0713b0 2023-12-12T09:53:51.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.575+0000 7f5b30cfd640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5b1006ae40 0x7f5b1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:51.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.575+0000 7f5b30cfd640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5b1006ae40 0x7f5b1006d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005880 tx=0x7f5b20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:51.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.579+0000 7f5b297fa640 1 -- 172.21.15.72:0/250474424 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b18076de0 con 0x7f5b2c0713b0 2023-12-12T09:53:51.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.772+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b2c074450 con 0x7f5b2c0713b0 2023-12-12T09:53:51.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.775+0000 7f5b297fa640 1 -- 172.21.15.72:0/250474424 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16774 (secure 0 0 0) 0x7f5b1807a400 con 0x7f5b2c0713b0 2023-12-12T09:53:51.777 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:51.778 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":38,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:53:48.818048+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:53:45.083566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-12T09:53:46.274070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:53:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5b1006ae40 msgr2=0x7f5b1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5b1006ae40 0x7f5b1006d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005880 tx=0x7f5b20009040 comp rx=0 tx=0).stop 2023-12-12T09:53:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 msgr2=0x7f5b2c151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c151de0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009890 tx=0x7f5b18004730 comp rx=0 tx=0).stop 2023-12-12T09:53:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 shutdown_connections 2023-12-12T09:53:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f5b1006ae40 0x7f5b1006d300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b2c159fd0 0x7f5b2c152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b2c0713b0 0x7f5b2c151de0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 --2- 172.21.15.72:0/250474424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b2c070750 0x7f5b2c143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 >> 172.21.15.72:0/250474424 conn(0x7f5b2c0b0860 msgr2=0x7f5b2c158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 shutdown_connections 2023-12-12T09:53:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:51.778+0000 7f5b31cff640 1 -- 172.21.15.72:0/250474424 wait complete. 2023-12-12T09:53:51.798 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-12T09:53:51.798 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:53:52.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.005+0000 7fdcd69c5640 1 Processor -- start 2023-12-12T09:53:52.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.005+0000 7fdcd69c5640 1 -- start start 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.007+0000 7fdcd69c5640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdcd0143990 0x7fdcd00cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.007+0000 7fdcd69c5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcd006a460 0x7fdcd006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.007+0000 7fdcd69c5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd006d520 con 0x7fdcd0071be0 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.007+0000 7fdcd69c5640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd01448d0 con 0x7fdcd0143990 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.007+0000 7fdcd69c5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd006ae10 con 0x7fdcd006a460 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.007+0000 7fdccffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdcd0143990 0x7fdcd00cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.007+0000 7fdccffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdcd0143990 0x7fdcd00cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52228/0 (socket says 172.21.15.72:52228) 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.007+0000 7fdccffff640 1 -- 172.21.15.72:0/2743672879 learned_addr learned my addr 172.21.15.72:0/2743672879 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.008+0000 7fdccffff640 1 -- 172.21.15.72:0/2743672879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcd006a460 msgr2=0x7fdcd006a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.008+0000 7fdccffff640 1 --2- 172.21.15.72:0/2743672879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcd006a460 0x7fdcd006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.008+0000 7fdccffff640 1 -- 172.21.15.72:0/2743672879 >> v1:172.21.15.113:6789/0 conn(0x7fdcd0071be0 legacy=0x7fdcd01441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:52.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.008+0000 7fdccffff640 1 -- 172.21.15.72:0/2743672879 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0143d70 con 0x7fdcd0143990 2023-12-12T09:53:52.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.009+0000 7fdccffff640 1 --2- 172.21.15.72:0/2743672879 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd0143990 0x7fdcd00cdec0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009e80 tx=0x7fdcc002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa149d1ad673565 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:52.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.009+0000 7fdccf7fe640 1 -- 172.21.15.72:0/2743672879 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc002ca50 con 0x7fdcd0143990 2023-12-12T09:53:52.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.009+0000 7fdccf7fe640 1 -- 172.21.15.72:0/2743672879 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc002cbb0 con 0x7fdcd0143990 2023-12-12T09:53:52.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.009+0000 7fdccf7fe640 1 -- 172.21.15.72:0/2743672879 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00343f0 con 0x7fdcd0143990 2023-12-12T09:53:52.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.010+0000 7fdcd69c5640 1 -- 172.21.15.72:0/2743672879 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd0143990 msgr2=0x7fdcd00cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.010+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/2743672879 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd0143990 0x7fdcd00cdec0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009e80 tx=0x7fdcc002beb0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.010+0000 7fdcd69c5640 1 -- 172.21.15.72:0/2743672879 shutdown_connections 2023-12-12T09:53:52.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.010+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/2743672879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcd006a460 0x7fdcd006a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.010+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/2743672879 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd0143990 0x7fdcd00cdec0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.010+0000 7fdcd69c5640 1 -- 172.21.15.72:0/2743672879 >> 172.21.15.72:0/2743672879 conn(0x7fdcd00cfe30 msgr2=0x7fdcd00d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:52.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.010+0000 7fdcd69c5640 1 -- 172.21.15.72:0/2743672879 shutdown_connections 2023-12-12T09:53:52.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.010+0000 7fdcd69c5640 1 -- 172.21.15.72:0/2743672879 wait complete. 2023-12-12T09:53:52.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.012+0000 7fdcd69c5640 1 Processor -- start 2023-12-12T09:53:52.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.012+0000 7fdcd69c5640 1 -- start start 2023-12-12T09:53:52.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdcd69c5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd006a460 0x7fdcd015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdcd69c5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcd0071be0 0x7fdcd013cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdcd69c5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcd0143990 0x7fdcd01f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdcd69c5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01f3eb0 con 0x7fdcd006a460 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdcd69c5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01f33e0 con 0x7fdcd0071be0 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdcd69c5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd013a100 con 0x7fdcd0143990 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdccffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd006a460 0x7fdcd015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdccffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd006a460 0x7fdcd015ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52244/0 (socket says 172.21.15.72:52244) 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdccffff640 1 -- 172.21.15.72:0/383360617 learned_addr learned my addr 172.21.15.72:0/383360617 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdccffff640 1 -- 172.21.15.72:0/383360617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcd0143990 msgr2=0x7fdcd01f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdccffff640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcd0143990 0x7fdcd01f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdccffff640 1 -- 172.21.15.72:0/383360617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcd0071be0 msgr2=0x7fdcd013cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdccffff640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcd0071be0 0x7fdcd013cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.013+0000 7fdccffff640 1 -- 172.21.15.72:0/383360617 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0009b30 con 0x7fdcd006a460 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.014+0000 7fdccffff640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd006a460 0x7fdcd015ce80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0036040 tx=0x7fdcc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.014+0000 7fdccd7fa640 1 -- 172.21.15.72:0/383360617 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0009510 con 0x7fdcd006a460 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.014+0000 7fdccd7fa640 1 -- 172.21.15.72:0/383360617 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc003e450 con 0x7fdcd006a460 2023-12-12T09:53:52.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.014+0000 7fdccd7fa640 1 -- 172.21.15.72:0/383360617 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc003e820 con 0x7fdcd006a460 2023-12-12T09:53:52.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.014+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd01f3610 con 0x7fdcd006a460 2023-12-12T09:53:52.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.014+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd01f3bd0 con 0x7fdcd006a460 2023-12-12T09:53:52.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.015+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd00745d0 con 0x7fdcd006a460 2023-12-12T09:53:52.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.016+0000 7fdccd7fa640 1 -- 172.21.15.72:0/383360617 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcc003e980 con 0x7fdcd006a460 2023-12-12T09:53:52.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.017+0000 7fdccd7fa640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcb006ae40 0x7fdcb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.017+0000 7fdccd7fa640 1 -- 172.21.15.72:0/383360617 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdcc00ac620 con 0x7fdcd006a460 2023-12-12T09:53:52.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.017+0000 7fdcc77fe640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcb006ae40 0x7fdcb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.018+0000 7fdcc77fe640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcb006ae40 0x7fdcb006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdcd013bc90 tx=0x7fdcb8007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.021+0000 7fdccd7fa640 1 -- 172.21.15.72:0/383360617 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdcc0077590 con 0x7fdcd006a460 2023-12-12T09:53:52.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.207+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdcd006da50 con 0x7fdcb006ae40 2023-12-12T09:53:52.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.224+0000 7fdccd7fa640 1 -- 172.21.15.72:0/383360617 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286716 (secure 0 0 0) 0x7fdcb80050d0 con 0x7fdcb006ae40 2023-12-12T09:53:52.230 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:52.232 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcb006ae40 msgr2=0x7fdcb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcb006ae40 0x7fdcb006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdcd013bc90 tx=0x7fdcb8007990 comp rx=0 tx=0).stop 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd006a460 msgr2=0x7fdcd015ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd006a460 0x7fdcd015ce80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0036040 tx=0x7fdcc0004a40 comp rx=0 tx=0).stop 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 shutdown_connections 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdcb006ae40 0x7fdcb006d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcd0143990 0x7fdcd01f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcd0071be0 0x7fdcd013cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 --2- 172.21.15.72:0/383360617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcd006a460 0x7fdcd015ce80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 >> 172.21.15.72:0/383360617 conn(0x7fdcd00cfe30 msgr2=0x7fdcd00cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 shutdown_connections 2023-12-12T09:53:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.231+0000 7fdcd69c5640 1 -- 172.21.15.72:0/383360617 wait complete. 2023-12-12T09:53:52.253 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2023-12-12T09:53:51.579460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220812,"kb_used_data":4600,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753908,"statfs":{"total":773094113280,"available":772868001792,"internally_reserved":0,"allocated":4710400,"data_stored":2670639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.801403"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.840643+0000","last_change":"2023-12-12T09:53:47.836470+0000","last_active":"2023-12-12T09:53:47.840643+0000","last_peered":"2023-12-12T09:53:47.840643+0000","last_clean":"2023-12-12T09:53:47.840643+0000","last_became_active":"2023-12-12T09:53:47.836207+0000","last_became_peered":"2023-12-12T09:53:47.836207+0000","last_unstale":"2023-12-12T09:53:47.840643+0000","last_undegraded":"2023-12-12T09:53:47.840643+0000","last_fullsized":"2023-12-12T09:53:47.840643+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:29.440308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.834189+0000","last_change":"2023-12-12T09:53:47.829820+0000","last_active":"2023-12-12T09:53:47.834189+0000","last_peered":"2023-12-12T09:53:47.834189+0000","last_clean":"2023-12-12T09:53:47.834189+0000","last_became_active":"2023-12-12T09:53:47.829480+0000","last_became_peered":"2023-12-12T09:53:47.829480+0000","last_unstale":"2023-12-12T09:53:47.834189+0000","last_undegraded":"2023-12-12T09:53:47.834189+0000","last_fullsized":"2023-12-12T09:53:47.834189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:26.252931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.849317+0000","last_change":"2023-12-12T09:53:47.843873+0000","last_active":"2023-12-12T09:53:47.849317+0000","last_peered":"2023-12-12T09:53:47.849317+0000","last_clean":"2023-12-12T09:53:47.849317+0000","last_became_active":"2023-12-12T09:53:47.842543+0000","last_became_peered":"2023-12-12T09:53:47.842543+0000","last_unstale":"2023-12-12T09:53:47.849317+0000","last_undegraded":"2023-12-12T09:53:47.849317+0000","last_fullsized":"2023-12-12T09:53:47.849317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:00:50.227315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978360+0000","last_change":"2023-12-12T09:53:47.838429+0000","last_active":"2023-12-12T09:53:48.978360+0000","last_peered":"2023-12-12T09:53:48.978360+0000","last_clean":"2023-12-12T09:53:48.978360+0000","last_became_active":"2023-12-12T09:53:47.837967+0000","last_became_peered":"2023-12-12T09:53:47.837967+0000","last_unstale":"2023-12-12T09:53:48.978360+0000","last_undegraded":"2023-12-12T09:53:48.978360+0000","last_fullsized":"2023-12-12T09:53:48.978360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:33:38.799302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978587+0000","last_change":"2023-12-12T09:53:47.829337+0000","last_active":"2023-12-12T09:53:48.978587+0000","last_peered":"2023-12-12T09:53:48.978587+0000","last_clean":"2023-12-12T09:53:48.978587+0000","last_became_active":"2023-12-12T09:53:47.828758+0000","last_became_peered":"2023-12-12T09:53:47.828758+0000","last_unstale":"2023-12-12T09:53:48.978587+0000","last_undegraded":"2023-12-12T09:53:48.978587+0000","last_fullsized":"2023-12-12T09:53:48.978587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:10:56.336106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.841092+0000","last_change":"2023-12-12T09:53:47.835002+0000","last_active":"2023-12-12T09:53:47.841092+0000","last_peered":"2023-12-12T09:53:47.841092+0000","last_clean":"2023-12-12T09:53:47.841092+0000","last_became_active":"2023-12-12T09:53:47.834734+0000","last_became_peered":"2023-12-12T09:53:47.834734+0000","last_unstale":"2023-12-12T09:53:47.841092+0000","last_undegraded":"2023-12-12T09:53:47.841092+0000","last_fullsized":"2023-12-12T09:53:47.841092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:05.777453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983477+0000","last_change":"2023-12-12T09:53:47.834358+0000","last_active":"2023-12-12T09:53:48.983477+0000","last_peered":"2023-12-12T09:53:48.983477+0000","last_clean":"2023-12-12T09:53:48.983477+0000","last_became_active":"2023-12-12T09:53:47.834049+0000","last_became_peered":"2023-12-12T09:53:47.834049+0000","last_unstale":"2023-12-12T09:53:48.983477+0000","last_undegraded":"2023-12-12T09:53:48.983477+0000","last_fullsized":"2023-12-12T09:53:48.983477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:33.797186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.834010+0000","last_change":"2023-12-12T09:53:47.826933+0000","last_active":"2023-12-12T09:53:47.834010+0000","last_peered":"2023-12-12T09:53:47.834010+0000","last_clean":"2023-12-12T09:53:47.834010+0000","last_became_active":"2023-12-12T09:53:47.825786+0000","last_became_peered":"2023-12-12T09:53:47.825786+0000","last_unstale":"2023-12-12T09:53:47.834010+0000","last_undegraded":"2023-12-12T09:53:47.834010+0000","last_fullsized":"2023-12-12T09:53:47.834010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T09:57:14.932430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.850961+0000","last_change":"2023-12-12T09:53:47.845438+0000","last_active":"2023-12-12T09:53:47.850961+0000","last_peered":"2023-12-12T09:53:47.850961+0000","last_clean":"2023-12-12T09:53:47.850961+0000","last_became_active":"2023-12-12T09:53:47.845134+0000","last_became_peered":"2023-12-12T09:53:47.845134+0000","last_unstale":"2023-12-12T09:53:47.850961+0000","last_undegraded":"2023-12-12T09:53:47.850961+0000","last_fullsized":"2023-12-12T09:53:47.850961+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:11.328087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984604+0000","last_change":"2023-12-12T09:53:47.841040+0000","last_active":"2023-12-12T09:53:48.984604+0000","last_peered":"2023-12-12T09:53:48.984604+0000","last_clean":"2023-12-12T09:53:48.984604+0000","last_became_active":"2023-12-12T09:53:47.840764+0000","last_became_peered":"2023-12-12T09:53:47.840764+0000","last_unstale":"2023-12-12T09:53:48.984604+0000","last_undegraded":"2023-12-12T09:53:48.984604+0000","last_fullsized":"2023-12-12T09:53:48.984604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:53:37.355846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984348+0000","last_change":"2023-12-12T09:53:47.832448+0000","last_active":"2023-12-12T09:53:48.984348+0000","last_peered":"2023-12-12T09:53:48.984348+0000","last_clean":"2023-12-12T09:53:48.984348+0000","last_became_active":"2023-12-12T09:53:47.831558+0000","last_became_peered":"2023-12-12T09:53:47.831558+0000","last_unstale":"2023-12-12T09:53:48.984348+0000","last_undegraded":"2023-12-12T09:53:48.984348+0000","last_fullsized":"2023-12-12T09:53:48.984348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:10:00.203063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.846118+0000","last_change":"2023-12-12T09:53:47.831511+0000","last_active":"2023-12-12T09:53:47.846118+0000","last_peered":"2023-12-12T09:53:47.846118+0000","last_clean":"2023-12-12T09:53:47.846118+0000","last_became_active":"2023-12-12T09:53:47.831122+0000","last_became_peered":"2023-12-12T09:53:47.831122+0000","last_unstale":"2023-12-12T09:53:47.846118+0000","last_undegraded":"2023-12-12T09:53:47.846118+0000","last_fullsized":"2023-12-12T09:53:47.846118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:23:53.334470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983622+0000","last_change":"2023-12-12T09:53:47.839398+0000","last_active":"2023-12-12T09:53:48.983622+0000","last_peered":"2023-12-12T09:53:48.983622+0000","last_clean":"2023-12-12T09:53:48.983622+0000","last_became_active":"2023-12-12T09:53:47.839133+0000","last_became_peered":"2023-12-12T09:53:47.839133+0000","last_unstale":"2023-12-12T09:53:48.983622+0000","last_undegraded":"2023-12-12T09:53:48.983622+0000","last_fullsized":"2023-12-12T09:53:48.983622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:46:25.818833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.187601+0000","last_change":"2023-12-12T09:53:47.829683+0000","last_active":"2023-12-12T09:53:49.187601+0000","last_peered":"2023-12-12T09:53:49.187601+0000","last_clean":"2023-12-12T09:53:49.187601+0000","last_became_active":"2023-12-12T09:53:47.828742+0000","last_became_peered":"2023-12-12T09:53:47.828742+0000","last_unstale":"2023-12-12T09:53:49.187601+0000","last_undegraded":"2023-12-12T09:53:49.187601+0000","last_fullsized":"2023-12-12T09:53:49.187601+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:24.286505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983542+0000","last_change":"2023-12-12T09:53:47.826402+0000","last_active":"2023-12-12T09:53:48.983542+0000","last_peered":"2023-12-12T09:53:48.983542+0000","last_clean":"2023-12-12T09:53:48.983542+0000","last_became_active":"2023-12-12T09:53:47.825514+0000","last_became_peered":"2023-12-12T09:53:47.825514+0000","last_unstale":"2023-12-12T09:53:48.983542+0000","last_undegraded":"2023-12-12T09:53:48.983542+0000","last_fullsized":"2023-12-12T09:53:48.983542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:29.440308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830640+0000","last_change":"2023-12-12T09:53:47.838169+0000","last_active":"2023-12-12T09:53:48.830640+0000","last_peered":"2023-12-12T09:53:48.830640+0000","last_clean":"2023-12-12T09:53:48.830640+0000","last_became_active":"2023-12-12T09:53:47.837516+0000","last_became_peered":"2023-12-12T09:53:47.837516+0000","last_unstale":"2023-12-12T09:53:48.830640+0000","last_undegraded":"2023-12-12T09:53:48.830640+0000","last_fullsized":"2023-12-12T09:53:48.830640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:48:01.111165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831192+0000","last_change":"2023-12-12T09:53:47.841930+0000","last_active":"2023-12-12T09:53:48.831192+0000","last_peered":"2023-12-12T09:53:48.831192+0000","last_clean":"2023-12-12T09:53:48.831192+0000","last_became_active":"2023-12-12T09:53:47.841609+0000","last_became_peered":"2023-12-12T09:53:47.841609+0000","last_unstale":"2023-12-12T09:53:48.831192+0000","last_undegraded":"2023-12-12T09:53:48.831192+0000","last_fullsized":"2023-12-12T09:53:48.831192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:21:23.166634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.829999+0000","last_change":"2023-12-12T09:53:47.839952+0000","last_active":"2023-12-12T09:53:48.829999+0000","last_peered":"2023-12-12T09:53:48.829999+0000","last_clean":"2023-12-12T09:53:48.829999+0000","last_became_active":"2023-12-12T09:53:47.831807+0000","last_became_peered":"2023-12-12T09:53:47.831807+0000","last_unstale":"2023-12-12T09:53:48.829999+0000","last_undegraded":"2023-12-12T09:53:48.829999+0000","last_fullsized":"2023-12-12T09:53:48.829999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:20.548700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978540+0000","last_change":"2023-12-12T09:53:47.836384+0000","last_active":"2023-12-12T09:53:48.978540+0000","last_peered":"2023-12-12T09:53:48.978540+0000","last_clean":"2023-12-12T09:53:48.978540+0000","last_became_active":"2023-12-12T09:53:47.836106+0000","last_became_peered":"2023-12-12T09:53:47.836106+0000","last_unstale":"2023-12-12T09:53:48.978540+0000","last_undegraded":"2023-12-12T09:53:48.978540+0000","last_fullsized":"2023-12-12T09:53:48.978540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:57.440955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831683+0000","last_change":"2023-12-12T09:53:47.832487+0000","last_active":"2023-12-12T09:53:48.831683+0000","last_peered":"2023-12-12T09:53:48.831683+0000","last_clean":"2023-12-12T09:53:48.831683+0000","last_became_active":"2023-12-12T09:53:47.832210+0000","last_became_peered":"2023-12-12T09:53:47.832210+0000","last_unstale":"2023-12-12T09:53:48.831683+0000","last_undegraded":"2023-12-12T09:53:48.831683+0000","last_fullsized":"2023-12-12T09:53:48.831683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:28.550271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984051+0000","last_change":"2023-12-12T09:53:47.832284+0000","last_active":"2023-12-12T09:53:48.984051+0000","last_peered":"2023-12-12T09:53:48.984051+0000","last_clean":"2023-12-12T09:53:48.984051+0000","last_became_active":"2023-12-12T09:53:47.831128+0000","last_became_peered":"2023-12-12T09:53:47.831128+0000","last_unstale":"2023-12-12T09:53:48.984051+0000","last_undegraded":"2023-12-12T09:53:48.984051+0000","last_fullsized":"2023-12-12T09:53:48.984051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:58:40.103609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188086+0000","last_change":"2023-12-12T09:53:47.831919+0000","last_active":"2023-12-12T09:53:49.188086+0000","last_peered":"2023-12-12T09:53:49.188086+0000","last_clean":"2023-12-12T09:53:49.188086+0000","last_became_active":"2023-12-12T09:53:47.829115+0000","last_became_peered":"2023-12-12T09:53:47.829115+0000","last_unstale":"2023-12-12T09:53:49.188086+0000","last_undegraded":"2023-12-12T09:53:49.188086+0000","last_fullsized":"2023-12-12T09:53:49.188086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T09:57:14.932430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830735+0000","last_change":"2023-12-12T09:53:47.843476+0000","last_active":"2023-12-12T09:53:48.830735+0000","last_peered":"2023-12-12T09:53:48.830735+0000","last_clean":"2023-12-12T09:53:48.830735+0000","last_became_active":"2023-12-12T09:53:47.842800+0000","last_became_peered":"2023-12-12T09:53:47.842800+0000","last_unstale":"2023-12-12T09:53:48.830735+0000","last_undegraded":"2023-12-12T09:53:48.830735+0000","last_fullsized":"2023-12-12T09:53:48.830735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:26.252931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978732+0000","last_change":"2023-12-12T09:53:47.832530+0000","last_active":"2023-12-12T09:53:48.978732+0000","last_peered":"2023-12-12T09:53:48.978732+0000","last_clean":"2023-12-12T09:53:48.978732+0000","last_became_active":"2023-12-12T09:53:47.832267+0000","last_became_peered":"2023-12-12T09:53:47.832267+0000","last_unstale":"2023-12-12T09:53:48.978732+0000","last_undegraded":"2023-12-12T09:53:48.978732+0000","last_fullsized":"2023-12-12T09:53:48.978732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:43:32.439493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830222+0000","last_change":"2023-12-12T09:53:47.835605+0000","last_active":"2023-12-12T09:53:48.830222+0000","last_peered":"2023-12-12T09:53:48.830222+0000","last_clean":"2023-12-12T09:53:48.830222+0000","last_became_active":"2023-12-12T09:53:47.835283+0000","last_became_peered":"2023-12-12T09:53:47.835283+0000","last_unstale":"2023-12-12T09:53:48.830222+0000","last_undegraded":"2023-12-12T09:53:48.830222+0000","last_fullsized":"2023-12-12T09:53:48.830222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:24.286505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.840993+0000","last_change":"2023-12-12T09:53:47.833508+0000","last_active":"2023-12-12T09:53:47.840993+0000","last_peered":"2023-12-12T09:53:47.840993+0000","last_clean":"2023-12-12T09:53:47.840993+0000","last_became_active":"2023-12-12T09:53:47.833243+0000","last_became_peered":"2023-12-12T09:53:47.833243+0000","last_unstale":"2023-12-12T09:53:47.840993+0000","last_undegraded":"2023-12-12T09:53:47.840993+0000","last_fullsized":"2023-12-12T09:53:47.840993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:44:11.637699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830363+0000","last_change":"2023-12-12T09:53:47.835408+0000","last_active":"2023-12-12T09:53:48.830363+0000","last_peered":"2023-12-12T09:53:48.830363+0000","last_clean":"2023-12-12T09:53:48.830363+0000","last_became_active":"2023-12-12T09:53:47.835000+0000","last_became_peered":"2023-12-12T09:53:47.835000+0000","last_unstale":"2023-12-12T09:53:48.830363+0000","last_undegraded":"2023-12-12T09:53:48.830363+0000","last_fullsized":"2023-12-12T09:53:48.830363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:53:42.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984158+0000","last_change":"2023-12-12T09:53:47.837046+0000","last_active":"2023-12-12T09:53:48.984158+0000","last_peered":"2023-12-12T09:53:48.984158+0000","last_clean":"2023-12-12T09:53:48.984158+0000","last_became_active":"2023-12-12T09:53:47.836708+0000","last_became_peered":"2023-12-12T09:53:47.836708+0000","last_unstale":"2023-12-12T09:53:48.984158+0000","last_undegraded":"2023-12-12T09:53:48.984158+0000","last_fullsized":"2023-12-12T09:53:48.984158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:03:35.377264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830037+0000","last_change":"2023-12-12T09:53:47.833134+0000","last_active":"2023-12-12T09:53:48.830037+0000","last_peered":"2023-12-12T09:53:48.830037+0000","last_clean":"2023-12-12T09:53:48.830037+0000","last_became_active":"2023-12-12T09:53:47.832842+0000","last_became_peered":"2023-12-12T09:53:47.832842+0000","last_unstale":"2023-12-12T09:53:48.830037+0000","last_undegraded":"2023-12-12T09:53:48.830037+0000","last_fullsized":"2023-12-12T09:53:48.830037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:11.328087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831022+0000","last_change":"2023-12-12T09:53:47.826921+0000","last_active":"2023-12-12T09:53:48.831022+0000","last_peered":"2023-12-12T09:53:48.831022+0000","last_clean":"2023-12-12T09:53:48.831022+0000","last_became_active":"2023-12-12T09:53:47.826625+0000","last_became_peered":"2023-12-12T09:53:47.826625+0000","last_unstale":"2023-12-12T09:53:48.831022+0000","last_undegraded":"2023-12-12T09:53:48.831022+0000","last_fullsized":"2023-12-12T09:53:48.831022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T09:57:14.932430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830661+0000","last_change":"2023-12-12T09:53:47.843667+0000","last_active":"2023-12-12T09:53:48.830661+0000","last_peered":"2023-12-12T09:53:48.830661+0000","last_clean":"2023-12-12T09:53:48.830661+0000","last_became_active":"2023-12-12T09:53:47.843178+0000","last_became_peered":"2023-12-12T09:53:47.843178+0000","last_unstale":"2023-12-12T09:53:48.830661+0000","last_undegraded":"2023-12-12T09:53:48.830661+0000","last_fullsized":"2023-12-12T09:53:48.830661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:18.340600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.840865+0000","last_change":"2023-12-12T09:53:47.831671+0000","last_active":"2023-12-12T09:53:47.840865+0000","last_peered":"2023-12-12T09:53:47.840865+0000","last_clean":"2023-12-12T09:53:47.840865+0000","last_became_active":"2023-12-12T09:53:47.831377+0000","last_became_peered":"2023-12-12T09:53:47.831377+0000","last_unstale":"2023-12-12T09:53:47.840865+0000","last_undegraded":"2023-12-12T09:53:47.840865+0000","last_fullsized":"2023-12-12T09:53:47.840865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:23:53.334470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830763+0000","last_change":"2023-12-12T09:53:47.843414+0000","last_active":"2023-12-12T09:53:48.830763+0000","last_peered":"2023-12-12T09:53:48.830763+0000","last_clean":"2023-12-12T09:53:48.830763+0000","last_became_active":"2023-12-12T09:53:47.843114+0000","last_became_peered":"2023-12-12T09:53:47.843114+0000","last_unstale":"2023-12-12T09:53:48.830763+0000","last_undegraded":"2023-12-12T09:53:48.830763+0000","last_fullsized":"2023-12-12T09:53:48.830763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:12.742806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830458+0000","last_change":"2023-12-12T09:53:46.831664+0000","last_active":"2023-12-12T09:53:48.830458+0000","last_peered":"2023-12-12T09:53:48.830458+0000","last_clean":"2023-12-12T09:53:48.830458+0000","last_became_active":"2023-12-12T09:53:46.831342+0000","last_became_peered":"2023-12-12T09:53:46.831342+0000","last_unstale":"2023-12-12T09:53:48.830458+0000","last_undegraded":"2023-12-12T09:53:48.830458+0000","last_fullsized":"2023-12-12T09:53:48.830458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:44:10.625305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.841767+0000","last_change":"2023-12-12T09:53:47.838090+0000","last_active":"2023-12-12T09:53:47.841767+0000","last_peered":"2023-12-12T09:53:47.841767+0000","last_clean":"2023-12-12T09:53:47.841767+0000","last_became_active":"2023-12-12T09:53:47.837705+0000","last_became_peered":"2023-12-12T09:53:47.837705+0000","last_unstale":"2023-12-12T09:53:47.841767+0000","last_undegraded":"2023-12-12T09:53:47.841767+0000","last_fullsized":"2023-12-12T09:53:47.841767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:00:50.227315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.859202+0000","last_change":"2023-12-12T09:53:46.819363+0000","last_active":"2023-12-12T09:53:47.859202+0000","last_peered":"2023-12-12T09:53:47.859202+0000","last_clean":"2023-12-12T09:53:47.859202+0000","last_became_active":"2023-12-12T09:53:46.819066+0000","last_became_peered":"2023-12-12T09:53:46.819066+0000","last_unstale":"2023-12-12T09:53:47.859202+0000","last_undegraded":"2023-12-12T09:53:47.859202+0000","last_fullsized":"2023-12-12T09:53:47.859202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:19.536305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188466+0000","last_change":"2023-12-12T09:53:47.841751+0000","last_active":"2023-12-12T09:53:49.188466+0000","last_peered":"2023-12-12T09:53:49.188466+0000","last_clean":"2023-12-12T09:53:49.188466+0000","last_became_active":"2023-12-12T09:53:47.839912+0000","last_became_peered":"2023-12-12T09:53:47.839912+0000","last_unstale":"2023-12-12T09:53:49.188466+0000","last_undegraded":"2023-12-12T09:53:49.188466+0000","last_fullsized":"2023-12-12T09:53:49.188466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:18.340600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.849655+0000","last_change":"2023-12-12T09:53:46.821382+0000","last_active":"2023-12-12T09:53:47.849655+0000","last_peered":"2023-12-12T09:53:47.849655+0000","last_clean":"2023-12-12T09:53:47.849655+0000","last_became_active":"2023-12-12T09:53:46.820805+0000","last_became_peered":"2023-12-12T09:53:46.820805+0000","last_unstale":"2023-12-12T09:53:47.849655+0000","last_undegraded":"2023-12-12T09:53:47.849655+0000","last_fullsized":"2023-12-12T09:53:47.849655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:19.536305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.831923+0000","last_change":"2023-12-12T09:53:47.824893+0000","last_active":"2023-12-12T09:53:47.831923+0000","last_peered":"2023-12-12T09:53:47.831923+0000","last_clean":"2023-12-12T09:53:47.831923+0000","last_became_active":"2023-12-12T09:53:47.824455+0000","last_became_peered":"2023-12-12T09:53:47.824455+0000","last_unstale":"2023-12-12T09:53:47.831923+0000","last_undegraded":"2023-12-12T09:53:47.831923+0000","last_fullsized":"2023-12-12T09:53:47.831923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:21:23.166634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188176+0000","last_change":"2023-12-12T09:53:46.824902+0000","last_active":"2023-12-12T09:53:49.188176+0000","last_peered":"2023-12-12T09:53:49.188176+0000","last_clean":"2023-12-12T09:53:49.188176+0000","last_became_active":"2023-12-12T09:53:46.820316+0000","last_became_peered":"2023-12-12T09:53:46.820316+0000","last_unstale":"2023-12-12T09:53:49.188176+0000","last_undegraded":"2023-12-12T09:53:49.188176+0000","last_fullsized":"2023-12-12T09:53:49.188176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:11:17.544738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.848858+0000","last_change":"2023-12-12T09:53:47.840807+0000","last_active":"2023-12-12T09:53:47.848858+0000","last_peered":"2023-12-12T09:53:47.848858+0000","last_clean":"2023-12-12T09:53:47.848858+0000","last_became_active":"2023-12-12T09:53:47.834899+0000","last_became_peered":"2023-12-12T09:53:47.834899+0000","last_unstale":"2023-12-12T09:53:47.848858+0000","last_undegraded":"2023-12-12T09:53:47.848858+0000","last_fullsized":"2023-12-12T09:53:47.848858+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:14:46.641653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983952+0000","last_change":"2023-12-12T09:53:46.821849+0000","last_active":"2023-12-12T09:53:48.983952+0000","last_peered":"2023-12-12T09:53:48.983952+0000","last_clean":"2023-12-12T09:53:48.983952+0000","last_became_active":"2023-12-12T09:53:46.821522+0000","last_became_peered":"2023-12-12T09:53:46.821522+0000","last_unstale":"2023-12-12T09:53:48.983952+0000","last_undegraded":"2023-12-12T09:53:48.983952+0000","last_fullsized":"2023-12-12T09:53:48.983952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:23:38.111445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983891+0000","last_change":"2023-12-12T09:53:47.837124+0000","last_active":"2023-12-12T09:53:48.983891+0000","last_peered":"2023-12-12T09:53:48.983891+0000","last_clean":"2023-12-12T09:53:48.983891+0000","last_became_active":"2023-12-12T09:53:47.836775+0000","last_became_peered":"2023-12-12T09:53:47.836775+0000","last_unstale":"2023-12-12T09:53:48.983891+0000","last_undegraded":"2023-12-12T09:53:48.983891+0000","last_fullsized":"2023-12-12T09:53:48.983891+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:14.541246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830518+0000","last_change":"2023-12-12T09:53:46.832964+0000","last_active":"2023-12-12T09:53:48.830518+0000","last_peered":"2023-12-12T09:53:48.830518+0000","last_clean":"2023-12-12T09:53:48.830518+0000","last_became_active":"2023-12-12T09:53:46.830772+0000","last_became_peered":"2023-12-12T09:53:46.830772+0000","last_unstale":"2023-12-12T09:53:48.830518+0000","last_undegraded":"2023-12-12T09:53:48.830518+0000","last_fullsized":"2023-12-12T09:53:48.830518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:10.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188240+0000","last_change":"2023-12-12T09:53:47.840670+0000","last_active":"2023-12-12T09:53:49.188240+0000","last_peered":"2023-12-12T09:53:49.188240+0000","last_clean":"2023-12-12T09:53:49.188240+0000","last_became_active":"2023-12-12T09:53:47.837955+0000","last_became_peered":"2023-12-12T09:53:47.837955+0000","last_unstale":"2023-12-12T09:53:49.188240+0000","last_undegraded":"2023-12-12T09:53:49.188240+0000","last_fullsized":"2023-12-12T09:53:49.188240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:28.550271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.816114+0000","last_change":"2023-12-12T09:53:46.828799+0000","last_active":"2023-12-12T09:53:47.816114+0000","last_peered":"2023-12-12T09:53:47.816114+0000","last_clean":"2023-12-12T09:53:47.816114+0000","last_became_active":"2023-12-12T09:53:46.828519+0000","last_became_peered":"2023-12-12T09:53:46.828519+0000","last_unstale":"2023-12-12T09:53:47.816114+0000","last_undegraded":"2023-12-12T09:53:47.816114+0000","last_fullsized":"2023-12-12T09:53:47.816114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:11:17.544738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.839505+0000","last_change":"2023-12-12T09:53:47.835361+0000","last_active":"2023-12-12T09:53:47.839505+0000","last_peered":"2023-12-12T09:53:47.839505+0000","last_clean":"2023-12-12T09:53:47.839505+0000","last_became_active":"2023-12-12T09:53:47.835076+0000","last_became_peered":"2023-12-12T09:53:47.835076+0000","last_unstale":"2023-12-12T09:53:47.839505+0000","last_undegraded":"2023-12-12T09:53:47.839505+0000","last_fullsized":"2023-12-12T09:53:47.839505+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:53:42.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.187405+0000","last_change":"2023-12-12T09:53:46.833689+0000","last_active":"2023-12-12T09:53:49.187405+0000","last_peered":"2023-12-12T09:53:49.187405+0000","last_clean":"2023-12-12T09:53:49.187405+0000","last_became_active":"2023-12-12T09:53:46.833289+0000","last_became_peered":"2023-12-12T09:53:46.833289+0000","last_unstale":"2023-12-12T09:53:49.187405+0000","last_undegraded":"2023-12-12T09:53:49.187405+0000","last_fullsized":"2023-12-12T09:53:49.187405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:43:47.743822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978520+0000","last_change":"2023-12-12T09:53:47.838254+0000","last_active":"2023-12-12T09:53:48.978520+0000","last_peered":"2023-12-12T09:53:48.978520+0000","last_clean":"2023-12-12T09:53:48.978520+0000","last_became_active":"2023-12-12T09:53:47.833762+0000","last_became_peered":"2023-12-12T09:53:47.833762+0000","last_unstale":"2023-12-12T09:53:48.978520+0000","last_undegraded":"2023-12-12T09:53:48.978520+0000","last_fullsized":"2023-12-12T09:53:48.978520+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:52:38.152266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.807661+0000","last_change":"2023-12-12T09:53:46.815733+0000","last_active":"2023-12-12T09:53:47.807661+0000","last_peered":"2023-12-12T09:53:47.807661+0000","last_clean":"2023-12-12T09:53:47.807661+0000","last_became_active":"2023-12-12T09:53:46.815440+0000","last_became_peered":"2023-12-12T09:53:46.815440+0000","last_unstale":"2023-12-12T09:53:47.807661+0000","last_undegraded":"2023-12-12T09:53:47.807661+0000","last_fullsized":"2023-12-12T09:53:47.807661+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:46:24.806438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830939+0000","last_change":"2023-12-12T09:53:47.848383+0000","last_active":"2023-12-12T09:53:48.830939+0000","last_peered":"2023-12-12T09:53:48.830939+0000","last_clean":"2023-12-12T09:53:48.830939+0000","last_became_active":"2023-12-12T09:53:47.846130+0000","last_became_peered":"2023-12-12T09:53:47.846130+0000","last_unstale":"2023-12-12T09:53:48.830939+0000","last_undegraded":"2023-12-12T09:53:48.830939+0000","last_fullsized":"2023-12-12T09:53:48.830939+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:33.797186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984867+0000","last_change":"2023-12-12T09:53:46.828640+0000","last_active":"2023-12-12T09:53:48.984867+0000","last_peered":"2023-12-12T09:53:48.984867+0000","last_clean":"2023-12-12T09:53:48.984867+0000","last_became_active":"2023-12-12T09:53:46.828353+0000","last_became_peered":"2023-12-12T09:53:46.828353+0000","last_unstale":"2023-12-12T09:53:48.984867+0000","last_undegraded":"2023-12-12T09:53:48.984867+0000","last_fullsized":"2023-12-12T09:53:48.984867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:48:00.098770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.851037+0000","last_change":"2023-12-12T09:53:47.845944+0000","last_active":"2023-12-12T09:53:47.851037+0000","last_peered":"2023-12-12T09:53:47.851037+0000","last_clean":"2023-12-12T09:53:47.851037+0000","last_became_active":"2023-12-12T09:53:47.845599+0000","last_became_peered":"2023-12-12T09:53:47.845599+0000","last_unstale":"2023-12-12T09:53:47.851037+0000","last_undegraded":"2023-12-12T09:53:47.851037+0000","last_fullsized":"2023-12-12T09:53:47.851037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:28.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984626+0000","last_change":"2023-12-12T09:53:46.826917+0000","last_active":"2023-12-12T09:53:48.984626+0000","last_peered":"2023-12-12T09:53:48.984626+0000","last_clean":"2023-12-12T09:53:48.984626+0000","last_became_active":"2023-12-12T09:53:46.826642+0000","last_became_peered":"2023-12-12T09:53:46.826642+0000","last_unstale":"2023-12-12T09:53:48.984626+0000","last_undegraded":"2023-12-12T09:53:48.984626+0000","last_fullsized":"2023-12-12T09:53:48.984626+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:17.328205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188620+0000","last_change":"2023-12-12T09:53:47.830875+0000","last_active":"2023-12-12T09:53:49.188620+0000","last_peered":"2023-12-12T09:53:49.188620+0000","last_clean":"2023-12-12T09:53:49.188620+0000","last_became_active":"2023-12-12T09:53:47.829956+0000","last_became_peered":"2023-12-12T09:53:47.829956+0000","last_unstale":"2023-12-12T09:53:49.188620+0000","last_undegraded":"2023-12-12T09:53:49.188620+0000","last_fullsized":"2023-12-12T09:53:49.188620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:48:01.111165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.860872+0000","last_change":"2023-12-12T09:53:46.818760+0000","last_active":"2023-12-12T09:53:47.860872+0000","last_peered":"2023-12-12T09:53:47.860872+0000","last_clean":"2023-12-12T09:53:47.860872+0000","last_became_active":"2023-12-12T09:53:46.818343+0000","last_became_peered":"2023-12-12T09:53:46.818343+0000","last_unstale":"2023-12-12T09:53:47.860872+0000","last_undegraded":"2023-12-12T09:53:47.860872+0000","last_fullsized":"2023-12-12T09:53:47.860872+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:23:38.111445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.848877+0000","last_change":"2023-12-12T09:53:47.844706+0000","last_active":"2023-12-12T09:53:47.848877+0000","last_peered":"2023-12-12T09:53:47.848877+0000","last_clean":"2023-12-12T09:53:47.848877+0000","last_became_active":"2023-12-12T09:53:47.844332+0000","last_became_peered":"2023-12-12T09:53:47.844332+0000","last_unstale":"2023-12-12T09:53:47.848877+0000","last_undegraded":"2023-12-12T09:53:47.848877+0000","last_fullsized":"2023-12-12T09:53:47.848877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:10:19.295434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.851457+0000","last_change":"2023-12-12T09:53:46.812322+0000","last_active":"2023-12-12T09:53:47.851457+0000","last_peered":"2023-12-12T09:53:47.851457+0000","last_clean":"2023-12-12T09:53:47.851457+0000","last_became_active":"2023-12-12T09:53:46.812053+0000","last_became_peered":"2023-12-12T09:53:46.812053+0000","last_unstale":"2023-12-12T09:53:47.851457+0000","last_undegraded":"2023-12-12T09:53:47.851457+0000","last_fullsized":"2023-12-12T09:53:47.851457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:55:01.516430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.839866+0000","last_change":"2023-12-12T09:53:47.832667+0000","last_active":"2023-12-12T09:53:47.839866+0000","last_peered":"2023-12-12T09:53:47.839866+0000","last_clean":"2023-12-12T09:53:47.839866+0000","last_became_active":"2023-12-12T09:53:47.832364+0000","last_became_peered":"2023-12-12T09:53:47.832364+0000","last_unstale":"2023-12-12T09:53:47.839866+0000","last_undegraded":"2023-12-12T09:53:47.839866+0000","last_fullsized":"2023-12-12T09:53:47.839866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:28.550271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.816297+0000","last_change":"2023-12-12T09:53:46.828674+0000","last_active":"2023-12-12T09:53:47.816297+0000","last_peered":"2023-12-12T09:53:47.816297+0000","last_clean":"2023-12-12T09:53:47.816297+0000","last_became_active":"2023-12-12T09:53:46.828409+0000","last_became_peered":"2023-12-12T09:53:46.828409+0000","last_unstale":"2023-12-12T09:53:47.816297+0000","last_undegraded":"2023-12-12T09:53:47.816297+0000","last_fullsized":"2023-12-12T09:53:47.816297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:14:45.629258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830851+0000","last_change":"2023-12-12T09:53:47.838530+0000","last_active":"2023-12-12T09:53:48.830851+0000","last_peered":"2023-12-12T09:53:48.830851+0000","last_clean":"2023-12-12T09:53:48.830851+0000","last_became_active":"2023-12-12T09:53:47.838219+0000","last_became_peered":"2023-12-12T09:53:47.838219+0000","last_unstale":"2023-12-12T09:53:48.830851+0000","last_undegraded":"2023-12-12T09:53:48.830851+0000","last_fullsized":"2023-12-12T09:53:48.830851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:55:02.528825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.829616+0000","last_change":"2023-12-12T09:53:46.830449+0000","last_active":"2023-12-12T09:53:48.829616+0000","last_peered":"2023-12-12T09:53:48.829616+0000","last_clean":"2023-12-12T09:53:48.829616+0000","last_became_active":"2023-12-12T09:53:46.830174+0000","last_became_peered":"2023-12-12T09:53:46.830174+0000","last_unstale":"2023-12-12T09:53:48.829616+0000","last_undegraded":"2023-12-12T09:53:48.829616+0000","last_fullsized":"2023-12-12T09:53:48.829616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:08:40.958883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.844239+0000","last_change":"2023-12-12T09:53:47.837719+0000","last_active":"2023-12-12T09:53:47.844239+0000","last_peered":"2023-12-12T09:53:47.844239+0000","last_clean":"2023-12-12T09:53:47.844239+0000","last_became_active":"2023-12-12T09:53:47.837436+0000","last_became_peered":"2023-12-12T09:53:47.837436+0000","last_unstale":"2023-12-12T09:53:47.844239+0000","last_undegraded":"2023-12-12T09:53:47.844239+0000","last_fullsized":"2023-12-12T09:53:47.844239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:48:01.111165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.829529+0000","last_change":"2023-12-12T09:53:46.831988+0000","last_active":"2023-12-12T09:53:48.829529+0000","last_peered":"2023-12-12T09:53:48.829529+0000","last_clean":"2023-12-12T09:53:48.829529+0000","last_became_active":"2023-12-12T09:53:46.831720+0000","last_became_peered":"2023-12-12T09:53:46.831720+0000","last_unstale":"2023-12-12T09:53:48.829529+0000","last_undegraded":"2023-12-12T09:53:48.829529+0000","last_fullsized":"2023-12-12T09:53:48.829529+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:00:49.214920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.832374+0000","last_change":"2023-12-12T09:53:47.821486+0000","last_active":"2023-12-12T09:53:47.832374+0000","last_peered":"2023-12-12T09:53:47.832374+0000","last_clean":"2023-12-12T09:53:47.832374+0000","last_became_active":"2023-12-12T09:53:47.821184+0000","last_became_peered":"2023-12-12T09:53:47.821184+0000","last_unstale":"2023-12-12T09:53:47.832374+0000","last_undegraded":"2023-12-12T09:53:47.832374+0000","last_fullsized":"2023-12-12T09:53:47.832374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:18.340600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984845+0000","last_change":"2023-12-12T09:53:46.832373+0000","last_active":"2023-12-12T09:53:48.984845+0000","last_peered":"2023-12-12T09:53:48.984845+0000","last_clean":"2023-12-12T09:53:48.984845+0000","last_became_active":"2023-12-12T09:53:46.830497+0000","last_became_peered":"2023-12-12T09:53:46.830497+0000","last_unstale":"2023-12-12T09:53:48.984845+0000","last_undegraded":"2023-12-12T09:53:48.984845+0000","last_fullsized":"2023-12-12T09:53:48.984845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:25.240536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984442+0000","last_change":"2023-12-12T09:53:47.834881+0000","last_active":"2023-12-12T09:53:48.984442+0000","last_peered":"2023-12-12T09:53:48.984442+0000","last_clean":"2023-12-12T09:53:48.984442+0000","last_became_active":"2023-12-12T09:53:47.834078+0000","last_became_peered":"2023-12-12T09:53:47.834078+0000","last_unstale":"2023-12-12T09:53:48.984442+0000","last_undegraded":"2023-12-12T09:53:48.984442+0000","last_fullsized":"2023-12-12T09:53:48.984442+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:34:42.332004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188478+0000","last_change":"2023-12-12T09:53:46.831524+0000","last_active":"2023-12-12T09:53:49.188478+0000","last_peered":"2023-12-12T09:53:49.188478+0000","last_clean":"2023-12-12T09:53:49.188478+0000","last_became_active":"2023-12-12T09:53:46.830628+0000","last_became_peered":"2023-12-12T09:53:46.830628+0000","last_unstale":"2023-12-12T09:53:49.188478+0000","last_undegraded":"2023-12-12T09:53:49.188478+0000","last_fullsized":"2023-12-12T09:53:49.188478+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:19.536305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188330+0000","last_change":"2023-12-12T09:53:47.833334+0000","last_active":"2023-12-12T09:53:49.188330+0000","last_peered":"2023-12-12T09:53:49.188330+0000","last_clean":"2023-12-12T09:53:49.188330+0000","last_became_active":"2023-12-12T09:53:47.832994+0000","last_became_peered":"2023-12-12T09:53:47.832994+0000","last_unstale":"2023-12-12T09:53:49.188330+0000","last_undegraded":"2023-12-12T09:53:49.188330+0000","last_fullsized":"2023-12-12T09:53:49.188330+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:23:39.123840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978526+0000","last_change":"2023-12-12T09:53:46.822712+0000","last_active":"2023-12-12T09:53:48.978526+0000","last_peered":"2023-12-12T09:53:48.978526+0000","last_clean":"2023-12-12T09:53:48.978526+0000","last_became_active":"2023-12-12T09:53:46.821700+0000","last_became_peered":"2023-12-12T09:53:46.821700+0000","last_unstale":"2023-12-12T09:53:48.978526+0000","last_undegraded":"2023-12-12T09:53:48.978526+0000","last_fullsized":"2023-12-12T09:53:48.978526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:00:47.182940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978811+0000","last_change":"2023-12-12T09:53:47.834751+0000","last_active":"2023-12-12T09:53:48.978811+0000","last_peered":"2023-12-12T09:53:48.978811+0000","last_clean":"2023-12-12T09:53:48.978811+0000","last_became_active":"2023-12-12T09:53:47.834435+0000","last_became_peered":"2023-12-12T09:53:47.834435+0000","last_unstale":"2023-12-12T09:53:48.978811+0000","last_undegraded":"2023-12-12T09:53:48.978811+0000","last_fullsized":"2023-12-12T09:53:48.978811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:29:49.006050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.816472+0000","last_change":"2023-12-12T09:53:46.821077+0000","last_active":"2023-12-12T09:53:47.816472+0000","last_peered":"2023-12-12T09:53:47.816472+0000","last_clean":"2023-12-12T09:53:47.816472+0000","last_became_active":"2023-12-12T09:53:46.820774+0000","last_became_peered":"2023-12-12T09:53:46.820774+0000","last_unstale":"2023-12-12T09:53:47.816472+0000","last_undegraded":"2023-12-12T09:53:47.816472+0000","last_fullsized":"2023-12-12T09:53:47.816472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:27.790822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831798+0000","last_change":"2023-12-12T09:53:47.838347+0000","last_active":"2023-12-12T09:53:48.831798+0000","last_peered":"2023-12-12T09:53:48.831798+0000","last_clean":"2023-12-12T09:53:48.831798+0000","last_became_active":"2023-12-12T09:53:47.837877+0000","last_became_peered":"2023-12-12T09:53:47.837877+0000","last_unstale":"2023-12-12T09:53:48.831798+0000","last_undegraded":"2023-12-12T09:53:48.831798+0000","last_fullsized":"2023-12-12T09:53:48.831798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:14.541246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978294+0000","last_change":"2023-12-12T09:53:46.814925+0000","last_active":"2023-12-12T09:53:48.978294+0000","last_peered":"2023-12-12T09:53:48.978294+0000","last_clean":"2023-12-12T09:53:48.978294+0000","last_became_active":"2023-12-12T09:53:46.814337+0000","last_became_peered":"2023-12-12T09:53:46.814337+0000","last_unstale":"2023-12-12T09:53:48.978294+0000","last_undegraded":"2023-12-12T09:53:48.978294+0000","last_fullsized":"2023-12-12T09:53:48.978294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:56.189089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.848805+0000","last_change":"2023-12-12T09:53:47.838780+0000","last_active":"2023-12-12T09:53:47.848805+0000","last_peered":"2023-12-12T09:53:47.848805+0000","last_clean":"2023-12-12T09:53:47.848805+0000","last_became_active":"2023-12-12T09:53:47.838516+0000","last_became_peered":"2023-12-12T09:53:47.838516+0000","last_unstale":"2023-12-12T09:53:47.848805+0000","last_undegraded":"2023-12-12T09:53:47.848805+0000","last_fullsized":"2023-12-12T09:53:47.848805+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:44:11.637699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978457+0000","last_change":"2023-12-12T09:53:46.823815+0000","last_active":"2023-12-12T09:53:48.978457+0000","last_peered":"2023-12-12T09:53:48.978457+0000","last_clean":"2023-12-12T09:53:48.978457+0000","last_became_active":"2023-12-12T09:53:46.823326+0000","last_became_peered":"2023-12-12T09:53:46.823326+0000","last_unstale":"2023-12-12T09:53:48.978457+0000","last_undegraded":"2023-12-12T09:53:48.978457+0000","last_fullsized":"2023-12-12T09:53:48.978457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:42.082271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983494+0000","last_change":"2023-12-12T09:53:47.825003+0000","last_active":"2023-12-12T09:53:48.983494+0000","last_peered":"2023-12-12T09:53:48.983494+0000","last_clean":"2023-12-12T09:53:48.983494+0000","last_became_active":"2023-12-12T09:53:47.824524+0000","last_became_peered":"2023-12-12T09:53:47.824524+0000","last_unstale":"2023-12-12T09:53:48.983494+0000","last_undegraded":"2023-12-12T09:53:48.983494+0000","last_fullsized":"2023-12-12T09:53:48.983494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:21:23.166634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830881+0000","last_change":"2023-12-12T09:53:46.834084+0000","last_active":"2023-12-12T09:53:48.830881+0000","last_peered":"2023-12-12T09:53:48.830881+0000","last_clean":"2023-12-12T09:53:48.830881+0000","last_became_active":"2023-12-12T09:53:46.833773+0000","last_became_peered":"2023-12-12T09:53:46.833773+0000","last_unstale":"2023-12-12T09:53:48.830881+0000","last_undegraded":"2023-12-12T09:53:48.830881+0000","last_fullsized":"2023-12-12T09:53:48.830881+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:44:10.625305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.839485+0000","last_change":"2023-12-12T09:53:47.835284+0000","last_active":"2023-12-12T09:53:47.839485+0000","last_peered":"2023-12-12T09:53:47.839485+0000","last_clean":"2023-12-12T09:53:47.839485+0000","last_became_active":"2023-12-12T09:53:47.834989+0000","last_became_peered":"2023-12-12T09:53:47.834989+0000","last_unstale":"2023-12-12T09:53:47.839485+0000","last_undegraded":"2023-12-12T09:53:47.839485+0000","last_fullsized":"2023-12-12T09:53:47.839485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:12.742806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978348+0000","last_change":"2023-12-12T09:53:46.812854+0000","last_active":"2023-12-12T09:53:48.978348+0000","last_peered":"2023-12-12T09:53:48.978348+0000","last_clean":"2023-12-12T09:53:48.978348+0000","last_became_active":"2023-12-12T09:53:46.812504+0000","last_became_peered":"2023-12-12T09:53:46.812504+0000","last_unstale":"2023-12-12T09:53:48.978348+0000","last_undegraded":"2023-12-12T09:53:48.978348+0000","last_fullsized":"2023-12-12T09:53:48.978348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:34:12.676764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.837714+0000","last_change":"2023-12-12T09:53:47.830923+0000","last_active":"2023-12-12T09:53:47.837714+0000","last_peered":"2023-12-12T09:53:47.837714+0000","last_clean":"2023-12-12T09:53:47.837714+0000","last_became_active":"2023-12-12T09:53:47.830631+0000","last_became_peered":"2023-12-12T09:53:47.830631+0000","last_unstale":"2023-12-12T09:53:47.837714+0000","last_undegraded":"2023-12-12T09:53:47.837714+0000","last_fullsized":"2023-12-12T09:53:47.837714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:10:19.295434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.985015+0000","last_change":"2023-12-12T09:53:46.833010+0000","last_active":"2023-12-12T09:53:48.985015+0000","last_peered":"2023-12-12T09:53:48.985015+0000","last_clean":"2023-12-12T09:53:48.985015+0000","last_became_active":"2023-12-12T09:53:46.829781+0000","last_became_peered":"2023-12-12T09:53:46.829781+0000","last_unstale":"2023-12-12T09:53:48.985015+0000","last_undegraded":"2023-12-12T09:53:48.985015+0000","last_fullsized":"2023-12-12T09:53:48.985015+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:11.730411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831370+0000","last_change":"2023-12-12T09:53:47.851831+0000","last_active":"2023-12-12T09:53:48.831370+0000","last_peered":"2023-12-12T09:53:48.831370+0000","last_clean":"2023-12-12T09:53:48.831370+0000","last_became_active":"2023-12-12T09:53:47.846500+0000","last_became_peered":"2023-12-12T09:53:47.846500+0000","last_unstale":"2023-12-12T09:53:48.831370+0000","last_undegraded":"2023-12-12T09:53:48.831370+0000","last_fullsized":"2023-12-12T09:53:48.831370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:29.440308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.981818+0000","last_change":"2023-12-12T09:53:46.827438+0000","last_active":"2023-12-12T09:53:48.981818+0000","last_peered":"2023-12-12T09:53:48.981818+0000","last_clean":"2023-12-12T09:53:48.981818+0000","last_became_active":"2023-12-12T09:53:46.827161+0000","last_became_peered":"2023-12-12T09:53:46.827161+0000","last_unstale":"2023-12-12T09:53:48.981818+0000","last_undegraded":"2023-12-12T09:53:48.981818+0000","last_fullsized":"2023-12-12T09:53:48.981818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:24:53.309495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.837272+0000","last_change":"2023-12-12T09:53:47.831845+0000","last_active":"2023-12-12T09:53:47.837272+0000","last_peered":"2023-12-12T09:53:47.837272+0000","last_clean":"2023-12-12T09:53:47.837272+0000","last_became_active":"2023-12-12T09:53:47.831516+0000","last_became_peered":"2023-12-12T09:53:47.831516+0000","last_unstale":"2023-12-12T09:53:47.837272+0000","last_undegraded":"2023-12-12T09:53:47.837272+0000","last_fullsized":"2023-12-12T09:53:47.837272+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:33.797186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.821902+0000","last_change":"2023-12-12T09:53:46.813712+0000","last_active":"2023-12-12T09:53:47.821902+0000","last_peered":"2023-12-12T09:53:47.821902+0000","last_clean":"2023-12-12T09:53:47.821902+0000","last_became_active":"2023-12-12T09:53:46.813410+0000","last_became_peered":"2023-12-12T09:53:46.813410+0000","last_unstale":"2023-12-12T09:53:47.821902+0000","last_undegraded":"2023-12-12T09:53:47.821902+0000","last_fullsized":"2023-12-12T09:53:47.821902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:11:17.544738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.848840+0000","last_change":"2023-12-12T09:53:47.835507+0000","last_active":"2023-12-12T09:53:47.848840+0000","last_peered":"2023-12-12T09:53:47.848840+0000","last_clean":"2023-12-12T09:53:47.848840+0000","last_became_active":"2023-12-12T09:53:47.834559+0000","last_became_peered":"2023-12-12T09:53:47.834559+0000","last_unstale":"2023-12-12T09:53:47.848840+0000","last_undegraded":"2023-12-12T09:53:47.848840+0000","last_fullsized":"2023-12-12T09:53:47.848840+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:05.777453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.825161+0000","last_change":"2023-12-12T09:53:46.830561+0000","last_active":"2023-12-12T09:53:47.825161+0000","last_peered":"2023-12-12T09:53:47.825161+0000","last_clean":"2023-12-12T09:53:47.825161+0000","last_became_active":"2023-12-12T09:53:46.830224+0000","last_became_peered":"2023-12-12T09:53:46.830224+0000","last_unstale":"2023-12-12T09:53:47.825161+0000","last_undegraded":"2023-12-12T09:53:47.825161+0000","last_fullsized":"2023-12-12T09:53:47.825161+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.834414+0000","last_change":"2023-12-12T09:53:47.829411+0000","last_active":"2023-12-12T09:53:47.834414+0000","last_peered":"2023-12-12T09:53:47.834414+0000","last_clean":"2023-12-12T09:53:47.834414+0000","last_became_active":"2023-12-12T09:53:47.829087+0000","last_became_peered":"2023-12-12T09:53:47.829087+0000","last_unstale":"2023-12-12T09:53:47.834414+0000","last_undegraded":"2023-12-12T09:53:47.834414+0000","last_fullsized":"2023-12-12T09:53:47.834414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:08:41.971277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831243+0000","last_change":"2023-12-12T09:53:46.824362+0000","last_active":"2023-12-12T09:53:48.831243+0000","last_peered":"2023-12-12T09:53:48.831243+0000","last_clean":"2023-12-12T09:53:48.831243+0000","last_became_active":"2023-12-12T09:53:46.824009+0000","last_became_peered":"2023-12-12T09:53:46.824009+0000","last_unstale":"2023-12-12T09:53:48.831243+0000","last_undegraded":"2023-12-12T09:53:48.831243+0000","last_fullsized":"2023-12-12T09:53:48.831243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:10:18.283040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.187614+0000","last_change":"2023-12-12T09:53:47.831689+0000","last_active":"2023-12-12T09:53:49.187614+0000","last_peered":"2023-12-12T09:53:49.187614+0000","last_clean":"2023-12-12T09:53:49.187614+0000","last_became_active":"2023-12-12T09:53:47.828381+0000","last_became_peered":"2023-12-12T09:53:47.828381+0000","last_unstale":"2023-12-12T09:53:49.187614+0000","last_undegraded":"2023-12-12T09:53:49.187614+0000","last_fullsized":"2023-12-12T09:53:49.187614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:53:42.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.187405+0000","last_change":"2023-12-12T09:53:46.824875+0000","last_active":"2023-12-12T09:53:49.187405+0000","last_peered":"2023-12-12T09:53:49.187405+0000","last_clean":"2023-12-12T09:53:49.187405+0000","last_became_active":"2023-12-12T09:53:46.824589+0000","last_became_peered":"2023-12-12T09:53:46.824589+0000","last_unstale":"2023-12-12T09:53:49.187405+0000","last_undegraded":"2023-12-12T09:53:49.187405+0000","last_fullsized":"2023-12-12T09:53:49.187405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:04.765059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831736+0000","last_change":"2023-12-12T09:53:47.847317+0000","last_active":"2023-12-12T09:53:48.831736+0000","last_peered":"2023-12-12T09:53:48.831736+0000","last_clean":"2023-12-12T09:53:48.831736+0000","last_became_active":"2023-12-12T09:53:47.846186+0000","last_became_peered":"2023-12-12T09:53:47.846186+0000","last_unstale":"2023-12-12T09:53:48.831736+0000","last_undegraded":"2023-12-12T09:53:48.831736+0000","last_fullsized":"2023-12-12T09:53:48.831736+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:46:25.818833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.802047+0000","last_change":"2023-12-12T09:53:46.814276+0000","last_active":"2023-12-12T09:53:47.802047+0000","last_peered":"2023-12-12T09:53:47.802047+0000","last_clean":"2023-12-12T09:53:47.802047+0000","last_became_active":"2023-12-12T09:53:46.813841+0000","last_became_peered":"2023-12-12T09:53:46.813841+0000","last_unstale":"2023-12-12T09:53:47.802047+0000","last_undegraded":"2023-12-12T09:53:47.802047+0000","last_fullsized":"2023-12-12T09:53:47.802047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984737+0000","last_change":"2023-12-12T09:53:47.833754+0000","last_active":"2023-12-12T09:53:48.984737+0000","last_peered":"2023-12-12T09:53:48.984737+0000","last_clean":"2023-12-12T09:53:48.984737+0000","last_became_active":"2023-12-12T09:53:47.833460+0000","last_became_peered":"2023-12-12T09:53:47.833460+0000","last_unstale":"2023-12-12T09:53:48.984737+0000","last_undegraded":"2023-12-12T09:53:48.984737+0000","last_fullsized":"2023-12-12T09:53:48.984737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:55.396686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830219+0000","last_change":"2023-12-12T09:53:46.826607+0000","last_active":"2023-12-12T09:53:48.830219+0000","last_peered":"2023-12-12T09:53:48.830219+0000","last_clean":"2023-12-12T09:53:48.830219+0000","last_became_active":"2023-12-12T09:53:46.826318+0000","last_became_peered":"2023-12-12T09:53:46.826318+0000","last_unstale":"2023-12-12T09:53:48.830219+0000","last_undegraded":"2023-12-12T09:53:48.830219+0000","last_fullsized":"2023-12-12T09:53:48.830219+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.821686+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:53:47.821686+0000","last_peered":"2023-12-12T09:53:47.821686+0000","last_clean":"2023-12-12T09:53:47.821686+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:53:47.821686+0000","last_undegraded":"2023-12-12T09:53:47.821686+0000","last_fullsized":"2023-12-12T09:53:47.821686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":780550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.85899999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.5509999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.306},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":6.6210000000000004,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.0129999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":1.619},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.371}]},{"osd":4,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.518}]},{"osd":5,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.585,"15min":1.585},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.5979999999999999,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.452}]},{"osd":6,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.633,"15min":1.633},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.044},{"interface":"front","average":{"1min":1.641,"5min":1.641,"15min":1.641},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.8300000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.4139999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":189082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.04},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.86899999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.381}]},{"osd":2,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.597}]},{"osd":3,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.4419999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":4.3209999999999997,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.5720000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.097},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":1.069,"5min":1.069,"15min":1.069},"max":{"1min":3.1989999999999998,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.1379999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":188612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":0.89200000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":3.8260000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.165},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.274},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.3100000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":3.944,"5min":3.944,"15min":3.944},"last":1.363},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.097}]},{"osd":4,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.462}]},{"osd":6,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.413},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.6060000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.1500000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.673,"15min":1.673},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.4950000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.79},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.96099999999999997}]},{"osd":1,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.079},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.6060000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.325},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.2629999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.47},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.167}]},{"osd":5,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"last":1.6899999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.7330000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.5229999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":1.863},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.74}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":183526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.627}]},{"osd":7,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.59499999999999997}]},{"osd":1,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.022}]},{"osd":3,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.95799999999999996}]},{"osd":4,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.482},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.2589999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.645}]},{"osd":6,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.3480000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.8130000000000002,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.599}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.876}]},{"osd":2,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.81499999999999995}]},{"osd":3,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.70999999999999996}]},{"osd":4,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":1.044}]},{"osd":5,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.95699999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":3.9209999999999998,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.0089999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":3.052,"5min":3.052,"15min":3.052},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.1369999999999996,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":1.222}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":771263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.0819999999999999,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":1.3420000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.8690000000000002,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":1.137}]},{"osd":3,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.9049999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.012}]},{"osd":4,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.577},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":1.4790000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.597},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.5109999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":1.5409999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":4.548,"5min":4.548,"15min":4.548},"last":1.385}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:53:52.260 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:53:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.511+0000 7f4d868e0640 1 Processor -- start 2023-12-12T09:53:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.512+0000 7f4d868e0640 1 -- start start 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.513+0000 7f4d868e0640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4d80072180 0x7f4d800b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.513+0000 7f4d868e0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4d8006c410 0x7f4d8015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.513+0000 7f4d868e0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d800b3bb0 con 0x7f4d80143e00 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.513+0000 7f4d868e0640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d801441e0 con 0x7f4d80072180 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.513+0000 7f4d868e0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d80072680 con 0x7f4d8006c410 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.514+0000 7f4d77fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4d80072180 0x7f4d800b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.514+0000 7f4d77fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4d80072180 0x7f4d800b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52254/0 (socket says 172.21.15.72:52254) 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.514+0000 7f4d77fff640 1 -- 172.21.15.72:0/352728148 learned_addr learned my addr 172.21.15.72:0/352728148 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.514+0000 7f4d84e7d640 1 --2- 172.21.15.72:0/352728148 >> v2:172.21.15.113:3300/0 conn(0x7f4d8006c410 0x7f4d8015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.514+0000 7f4d77fff640 1 -- 172.21.15.72:0/352728148 >> v2:172.21.15.113:3300/0 conn(0x7f4d8006c410 msgr2=0x7f4d8015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.514+0000 7f4d77fff640 1 --2- 172.21.15.72:0/352728148 >> v2:172.21.15.113:3300/0 conn(0x7f4d8006c410 0x7f4d8015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.514+0000 7f4d77fff640 1 -- 172.21.15.72:0/352728148 >> v1:172.21.15.113:6789/0 conn(0x7f4d80143e00 legacy=0x7f4d800b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:52.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.514+0000 7f4d77fff640 1 -- 172.21.15.72:0/352728148 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800b1e30 con 0x7f4d80072180 2023-12-12T09:53:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.515+0000 7f4d77fff640 1 --2- 172.21.15.72:0/352728148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80072180 0x7f4d800b4750 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4d80072b60 tx=0x7f4d6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b9bd6ebda0f876 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d7f7fe640 1 -- 172.21.15.72:0/352728148 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68031870 con 0x7f4d80072180 2023-12-12T09:53:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d7f7fe640 1 -- 172.21.15.72:0/352728148 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d680319d0 con 0x7f4d80072180 2023-12-12T09:53:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d7f7fe640 1 -- 172.21.15.72:0/352728148 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68031d60 con 0x7f4d80072180 2023-12-12T09:53:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d868e0640 1 -- 172.21.15.72:0/352728148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80072180 msgr2=0x7f4d800b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d868e0640 1 --2- 172.21.15.72:0/352728148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80072180 0x7f4d800b4750 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4d80072b60 tx=0x7f4d6802b910 comp rx=0 tx=0).stop 2023-12-12T09:53:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d868e0640 1 -- 172.21.15.72:0/352728148 shutdown_connections 2023-12-12T09:53:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d868e0640 1 --2- 172.21.15.72:0/352728148 >> v2:172.21.15.113:3300/0 conn(0x7f4d8006c410 0x7f4d8015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d868e0640 1 --2- 172.21.15.72:0/352728148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80072180 0x7f4d800b4750 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.516+0000 7f4d868e0640 1 -- 172.21.15.72:0/352728148 >> 172.21.15.72:0/352728148 conn(0x7f4d800b0a90 msgr2=0x7f4d800b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.517+0000 7f4d868e0640 1 -- 172.21.15.72:0/352728148 shutdown_connections 2023-12-12T09:53:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.517+0000 7f4d868e0640 1 -- 172.21.15.72:0/352728148 wait complete. 2023-12-12T09:53:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.518+0000 7f4d868e0640 1 Processor -- start 2023-12-12T09:53:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.518+0000 7f4d868e0640 1 -- start start 2023-12-12T09:53:52.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.519+0000 7f4d868e0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8006c410 0x7f4d80157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.519+0000 7f4d868e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d80072180 0x7f4d80141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.519+0000 7f4d868e0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80143e00 0x7f4d801f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.519+0000 7f4d868e0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80142420 con 0x7f4d80143e00 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.519+0000 7f4d868e0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d801426a0 con 0x7f4d8006c410 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.519+0000 7f4d868e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80142980 con 0x7f4d80072180 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d84e7d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80143e00 0x7f4d801f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d84e7d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80143e00 0x7f4d801f2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52262/0 (socket says 172.21.15.72:52262) 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d84e7d640 1 -- 172.21.15.72:0/726542799 learned_addr learned my addr 172.21.15.72:0/726542799 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d7ffff640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8006c410 0x7f4d80157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d77fff640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d80072180 0x7f4d80141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d84e7d640 1 -- 172.21.15.72:0/726542799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d80072180 msgr2=0x7f4d80141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d84e7d640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d80072180 0x7f4d80141ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d84e7d640 1 -- 172.21.15.72:0/726542799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8006c410 msgr2=0x7f4d80157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d84e7d640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8006c410 0x7f4d80157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.520+0000 7f4d84e7d640 1 -- 172.21.15.72:0/726542799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d680094d0 con 0x7f4d80143e00 2023-12-12T09:53:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.521+0000 7f4d84e7d640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80143e00 0x7f4d801f2570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007e40 tx=0x7f4d7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.521+0000 7f4d7d7fa640 1 -- 172.21.15.72:0/726542799 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78014620 con 0x7f4d80143e00 2023-12-12T09:53:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.521+0000 7f4d7d7fa640 1 -- 172.21.15.72:0/726542799 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7800ee90 con 0x7f4d80143e00 2023-12-12T09:53:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.521+0000 7f4d7d7fa640 1 -- 172.21.15.72:0/726542799 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d780149e0 con 0x7f4d80143e00 2023-12-12T09:53:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.521+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d80142c60 con 0x7f4d80143e00 2023-12-12T09:53:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.521+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d801f2ab0 con 0x7f4d80143e00 2023-12-12T09:53:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.522+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d801f2ca0 con 0x7f4d80143e00 2023-12-12T09:53:52.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.523+0000 7f4d7d7fa640 1 -- 172.21.15.72:0/726542799 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d78014b40 con 0x7f4d80143e00 2023-12-12T09:53:52.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.524+0000 7f4d7d7fa640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d6006ae40 0x7f4d6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:52.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.524+0000 7f4d7d7fa640 1 -- 172.21.15.72:0/726542799 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4d7808a140 con 0x7f4d80143e00 2023-12-12T09:53:52.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.524+0000 7f4d7ffff640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d6006ae40 0x7f4d6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:52.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.525+0000 7f4d7ffff640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d6006ae40 0x7f4d6006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006fd0 tx=0x7f4d70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:52.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.528+0000 7f4d7d7fa640 1 -- 172.21.15.72:0/726542799 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d780550e0 con 0x7f4d80143e00 2023-12-12T09:53:52.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.715+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d800b23f0 con 0x7f4d6006ae40 2023-12-12T09:53:52.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.738+0000 7f4d7d7fa640 1 -- 172.21.15.72:0/726542799 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286716 (secure 0 0 0) 0x7f4d7000be30 con 0x7f4d6006ae40 2023-12-12T09:53:52.743 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:52.745 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:53:52.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d6006ae40 msgr2=0x7f4d6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d6006ae40 0x7f4d6006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006fd0 tx=0x7f4d70008040 comp rx=0 tx=0).stop 2023-12-12T09:53:52.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80143e00 msgr2=0x7f4d801f2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:52.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80143e00 0x7f4d801f2570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007e40 tx=0x7f4d7800e4b0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 shutdown_connections 2023-12-12T09:53:52.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4d6006ae40 0x7f4d6006d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80143e00 0x7f4d801f2570 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d80072180 0x7f4d80141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 --2- 172.21.15.72:0/726542799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8006c410 0x7f4d80157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:52.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.743+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 >> 172.21.15.72:0/726542799 conn(0x7f4d800b0a90 msgr2=0x7f4d80070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:52.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.744+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 shutdown_connections 2023-12-12T09:53:52.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:52.744+0000 7f4d868e0640 1 -- 172.21.15.72:0/726542799 wait complete. 2023-12-12T09:53:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2023-12-12T09:53:51.579460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220812,"kb_used_data":4600,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753908,"statfs":{"total":773094113280,"available":772868001792,"internally_reserved":0,"allocated":4710400,"data_stored":2670639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.801403"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.840643+0000","last_change":"2023-12-12T09:53:47.836470+0000","last_active":"2023-12-12T09:53:47.840643+0000","last_peered":"2023-12-12T09:53:47.840643+0000","last_clean":"2023-12-12T09:53:47.840643+0000","last_became_active":"2023-12-12T09:53:47.836207+0000","last_became_peered":"2023-12-12T09:53:47.836207+0000","last_unstale":"2023-12-12T09:53:47.840643+0000","last_undegraded":"2023-12-12T09:53:47.840643+0000","last_fullsized":"2023-12-12T09:53:47.840643+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:29.440308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.834189+0000","last_change":"2023-12-12T09:53:47.829820+0000","last_active":"2023-12-12T09:53:47.834189+0000","last_peered":"2023-12-12T09:53:47.834189+0000","last_clean":"2023-12-12T09:53:47.834189+0000","last_became_active":"2023-12-12T09:53:47.829480+0000","last_became_peered":"2023-12-12T09:53:47.829480+0000","last_unstale":"2023-12-12T09:53:47.834189+0000","last_undegraded":"2023-12-12T09:53:47.834189+0000","last_fullsized":"2023-12-12T09:53:47.834189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:26.252931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.849317+0000","last_change":"2023-12-12T09:53:47.843873+0000","last_active":"2023-12-12T09:53:47.849317+0000","last_peered":"2023-12-12T09:53:47.849317+0000","last_clean":"2023-12-12T09:53:47.849317+0000","last_became_active":"2023-12-12T09:53:47.842543+0000","last_became_peered":"2023-12-12T09:53:47.842543+0000","last_unstale":"2023-12-12T09:53:47.849317+0000","last_undegraded":"2023-12-12T09:53:47.849317+0000","last_fullsized":"2023-12-12T09:53:47.849317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:00:50.227315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978360+0000","last_change":"2023-12-12T09:53:47.838429+0000","last_active":"2023-12-12T09:53:48.978360+0000","last_peered":"2023-12-12T09:53:48.978360+0000","last_clean":"2023-12-12T09:53:48.978360+0000","last_became_active":"2023-12-12T09:53:47.837967+0000","last_became_peered":"2023-12-12T09:53:47.837967+0000","last_unstale":"2023-12-12T09:53:48.978360+0000","last_undegraded":"2023-12-12T09:53:48.978360+0000","last_fullsized":"2023-12-12T09:53:48.978360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:33:38.799302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978587+0000","last_change":"2023-12-12T09:53:47.829337+0000","last_active":"2023-12-12T09:53:48.978587+0000","last_peered":"2023-12-12T09:53:48.978587+0000","last_clean":"2023-12-12T09:53:48.978587+0000","last_became_active":"2023-12-12T09:53:47.828758+0000","last_became_peered":"2023-12-12T09:53:47.828758+0000","last_unstale":"2023-12-12T09:53:48.978587+0000","last_undegraded":"2023-12-12T09:53:48.978587+0000","last_fullsized":"2023-12-12T09:53:48.978587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:10:56.336106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.841092+0000","last_change":"2023-12-12T09:53:47.835002+0000","last_active":"2023-12-12T09:53:47.841092+0000","last_peered":"2023-12-12T09:53:47.841092+0000","last_clean":"2023-12-12T09:53:47.841092+0000","last_became_active":"2023-12-12T09:53:47.834734+0000","last_became_peered":"2023-12-12T09:53:47.834734+0000","last_unstale":"2023-12-12T09:53:47.841092+0000","last_undegraded":"2023-12-12T09:53:47.841092+0000","last_fullsized":"2023-12-12T09:53:47.841092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:05.777453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983477+0000","last_change":"2023-12-12T09:53:47.834358+0000","last_active":"2023-12-12T09:53:48.983477+0000","last_peered":"2023-12-12T09:53:48.983477+0000","last_clean":"2023-12-12T09:53:48.983477+0000","last_became_active":"2023-12-12T09:53:47.834049+0000","last_became_peered":"2023-12-12T09:53:47.834049+0000","last_unstale":"2023-12-12T09:53:48.983477+0000","last_undegraded":"2023-12-12T09:53:48.983477+0000","last_fullsized":"2023-12-12T09:53:48.983477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:33.797186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.834010+0000","last_change":"2023-12-12T09:53:47.826933+0000","last_active":"2023-12-12T09:53:47.834010+0000","last_peered":"2023-12-12T09:53:47.834010+0000","last_clean":"2023-12-12T09:53:47.834010+0000","last_became_active":"2023-12-12T09:53:47.825786+0000","last_became_peered":"2023-12-12T09:53:47.825786+0000","last_unstale":"2023-12-12T09:53:47.834010+0000","last_undegraded":"2023-12-12T09:53:47.834010+0000","last_fullsized":"2023-12-12T09:53:47.834010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T09:57:14.932430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.850961+0000","last_change":"2023-12-12T09:53:47.845438+0000","last_active":"2023-12-12T09:53:47.850961+0000","last_peered":"2023-12-12T09:53:47.850961+0000","last_clean":"2023-12-12T09:53:47.850961+0000","last_became_active":"2023-12-12T09:53:47.845134+0000","last_became_peered":"2023-12-12T09:53:47.845134+0000","last_unstale":"2023-12-12T09:53:47.850961+0000","last_undegraded":"2023-12-12T09:53:47.850961+0000","last_fullsized":"2023-12-12T09:53:47.850961+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:11.328087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984604+0000","last_change":"2023-12-12T09:53:47.841040+0000","last_active":"2023-12-12T09:53:48.984604+0000","last_peered":"2023-12-12T09:53:48.984604+0000","last_clean":"2023-12-12T09:53:48.984604+0000","last_became_active":"2023-12-12T09:53:47.840764+0000","last_became_peered":"2023-12-12T09:53:47.840764+0000","last_unstale":"2023-12-12T09:53:48.984604+0000","last_undegraded":"2023-12-12T09:53:48.984604+0000","last_fullsized":"2023-12-12T09:53:48.984604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:53:37.355846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984348+0000","last_change":"2023-12-12T09:53:47.832448+0000","last_active":"2023-12-12T09:53:48.984348+0000","last_peered":"2023-12-12T09:53:48.984348+0000","last_clean":"2023-12-12T09:53:48.984348+0000","last_became_active":"2023-12-12T09:53:47.831558+0000","last_became_peered":"2023-12-12T09:53:47.831558+0000","last_unstale":"2023-12-12T09:53:48.984348+0000","last_undegraded":"2023-12-12T09:53:48.984348+0000","last_fullsized":"2023-12-12T09:53:48.984348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:10:00.203063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.846118+0000","last_change":"2023-12-12T09:53:47.831511+0000","last_active":"2023-12-12T09:53:47.846118+0000","last_peered":"2023-12-12T09:53:47.846118+0000","last_clean":"2023-12-12T09:53:47.846118+0000","last_became_active":"2023-12-12T09:53:47.831122+0000","last_became_peered":"2023-12-12T09:53:47.831122+0000","last_unstale":"2023-12-12T09:53:47.846118+0000","last_undegraded":"2023-12-12T09:53:47.846118+0000","last_fullsized":"2023-12-12T09:53:47.846118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:23:53.334470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983622+0000","last_change":"2023-12-12T09:53:47.839398+0000","last_active":"2023-12-12T09:53:48.983622+0000","last_peered":"2023-12-12T09:53:48.983622+0000","last_clean":"2023-12-12T09:53:48.983622+0000","last_became_active":"2023-12-12T09:53:47.839133+0000","last_became_peered":"2023-12-12T09:53:47.839133+0000","last_unstale":"2023-12-12T09:53:48.983622+0000","last_undegraded":"2023-12-12T09:53:48.983622+0000","last_fullsized":"2023-12-12T09:53:48.983622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:46:25.818833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.187601+0000","last_change":"2023-12-12T09:53:47.829683+0000","last_active":"2023-12-12T09:53:49.187601+0000","last_peered":"2023-12-12T09:53:49.187601+0000","last_clean":"2023-12-12T09:53:49.187601+0000","last_became_active":"2023-12-12T09:53:47.828742+0000","last_became_peered":"2023-12-12T09:53:47.828742+0000","last_unstale":"2023-12-12T09:53:49.187601+0000","last_undegraded":"2023-12-12T09:53:49.187601+0000","last_fullsized":"2023-12-12T09:53:49.187601+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:24.286505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983542+0000","last_change":"2023-12-12T09:53:47.826402+0000","last_active":"2023-12-12T09:53:48.983542+0000","last_peered":"2023-12-12T09:53:48.983542+0000","last_clean":"2023-12-12T09:53:48.983542+0000","last_became_active":"2023-12-12T09:53:47.825514+0000","last_became_peered":"2023-12-12T09:53:47.825514+0000","last_unstale":"2023-12-12T09:53:48.983542+0000","last_undegraded":"2023-12-12T09:53:48.983542+0000","last_fullsized":"2023-12-12T09:53:48.983542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:29.440308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830640+0000","last_change":"2023-12-12T09:53:47.838169+0000","last_active":"2023-12-12T09:53:48.830640+0000","last_peered":"2023-12-12T09:53:48.830640+0000","last_clean":"2023-12-12T09:53:48.830640+0000","last_became_active":"2023-12-12T09:53:47.837516+0000","last_became_peered":"2023-12-12T09:53:47.837516+0000","last_unstale":"2023-12-12T09:53:48.830640+0000","last_undegraded":"2023-12-12T09:53:48.830640+0000","last_fullsized":"2023-12-12T09:53:48.830640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:48:01.111165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831192+0000","last_change":"2023-12-12T09:53:47.841930+0000","last_active":"2023-12-12T09:53:48.831192+0000","last_peered":"2023-12-12T09:53:48.831192+0000","last_clean":"2023-12-12T09:53:48.831192+0000","last_became_active":"2023-12-12T09:53:47.841609+0000","last_became_peered":"2023-12-12T09:53:47.841609+0000","last_unstale":"2023-12-12T09:53:48.831192+0000","last_undegraded":"2023-12-12T09:53:48.831192+0000","last_fullsized":"2023-12-12T09:53:48.831192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:21:23.166634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.829999+0000","last_change":"2023-12-12T09:53:47.839952+0000","last_active":"2023-12-12T09:53:48.829999+0000","last_peered":"2023-12-12T09:53:48.829999+0000","last_clean":"2023-12-12T09:53:48.829999+0000","last_became_active":"2023-12-12T09:53:47.831807+0000","last_became_peered":"2023-12-12T09:53:47.831807+0000","last_unstale":"2023-12-12T09:53:48.829999+0000","last_undegraded":"2023-12-12T09:53:48.829999+0000","last_fullsized":"2023-12-12T09:53:48.829999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:20.548700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978540+0000","last_change":"2023-12-12T09:53:47.836384+0000","last_active":"2023-12-12T09:53:48.978540+0000","last_peered":"2023-12-12T09:53:48.978540+0000","last_clean":"2023-12-12T09:53:48.978540+0000","last_became_active":"2023-12-12T09:53:47.836106+0000","last_became_peered":"2023-12-12T09:53:47.836106+0000","last_unstale":"2023-12-12T09:53:48.978540+0000","last_undegraded":"2023-12-12T09:53:48.978540+0000","last_fullsized":"2023-12-12T09:53:48.978540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:57.440955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831683+0000","last_change":"2023-12-12T09:53:47.832487+0000","last_active":"2023-12-12T09:53:48.831683+0000","last_peered":"2023-12-12T09:53:48.831683+0000","last_clean":"2023-12-12T09:53:48.831683+0000","last_became_active":"2023-12-12T09:53:47.832210+0000","last_became_peered":"2023-12-12T09:53:47.832210+0000","last_unstale":"2023-12-12T09:53:48.831683+0000","last_undegraded":"2023-12-12T09:53:48.831683+0000","last_fullsized":"2023-12-12T09:53:48.831683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:28.550271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984051+0000","last_change":"2023-12-12T09:53:47.832284+0000","last_active":"2023-12-12T09:53:48.984051+0000","last_peered":"2023-12-12T09:53:48.984051+0000","last_clean":"2023-12-12T09:53:48.984051+0000","last_became_active":"2023-12-12T09:53:47.831128+0000","last_became_peered":"2023-12-12T09:53:47.831128+0000","last_unstale":"2023-12-12T09:53:48.984051+0000","last_undegraded":"2023-12-12T09:53:48.984051+0000","last_fullsized":"2023-12-12T09:53:48.984051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:58:40.103609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188086+0000","last_change":"2023-12-12T09:53:47.831919+0000","last_active":"2023-12-12T09:53:49.188086+0000","last_peered":"2023-12-12T09:53:49.188086+0000","last_clean":"2023-12-12T09:53:49.188086+0000","last_became_active":"2023-12-12T09:53:47.829115+0000","last_became_peered":"2023-12-12T09:53:47.829115+0000","last_unstale":"2023-12-12T09:53:49.188086+0000","last_undegraded":"2023-12-12T09:53:49.188086+0000","last_fullsized":"2023-12-12T09:53:49.188086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T09:57:14.932430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830735+0000","last_change":"2023-12-12T09:53:47.843476+0000","last_active":"2023-12-12T09:53:48.830735+0000","last_peered":"2023-12-12T09:53:48.830735+0000","last_clean":"2023-12-12T09:53:48.830735+0000","last_became_active":"2023-12-12T09:53:47.842800+0000","last_became_peered":"2023-12-12T09:53:47.842800+0000","last_unstale":"2023-12-12T09:53:48.830735+0000","last_undegraded":"2023-12-12T09:53:48.830735+0000","last_fullsized":"2023-12-12T09:53:48.830735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:26.252931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978732+0000","last_change":"2023-12-12T09:53:47.832530+0000","last_active":"2023-12-12T09:53:48.978732+0000","last_peered":"2023-12-12T09:53:48.978732+0000","last_clean":"2023-12-12T09:53:48.978732+0000","last_became_active":"2023-12-12T09:53:47.832267+0000","last_became_peered":"2023-12-12T09:53:47.832267+0000","last_unstale":"2023-12-12T09:53:48.978732+0000","last_undegraded":"2023-12-12T09:53:48.978732+0000","last_fullsized":"2023-12-12T09:53:48.978732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:43:32.439493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830222+0000","last_change":"2023-12-12T09:53:47.835605+0000","last_active":"2023-12-12T09:53:48.830222+0000","last_peered":"2023-12-12T09:53:48.830222+0000","last_clean":"2023-12-12T09:53:48.830222+0000","last_became_active":"2023-12-12T09:53:47.835283+0000","last_became_peered":"2023-12-12T09:53:47.835283+0000","last_unstale":"2023-12-12T09:53:48.830222+0000","last_undegraded":"2023-12-12T09:53:48.830222+0000","last_fullsized":"2023-12-12T09:53:48.830222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:24.286505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.840993+0000","last_change":"2023-12-12T09:53:47.833508+0000","last_active":"2023-12-12T09:53:47.840993+0000","last_peered":"2023-12-12T09:53:47.840993+0000","last_clean":"2023-12-12T09:53:47.840993+0000","last_became_active":"2023-12-12T09:53:47.833243+0000","last_became_peered":"2023-12-12T09:53:47.833243+0000","last_unstale":"2023-12-12T09:53:47.840993+0000","last_undegraded":"2023-12-12T09:53:47.840993+0000","last_fullsized":"2023-12-12T09:53:47.840993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:44:11.637699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830363+0000","last_change":"2023-12-12T09:53:47.835408+0000","last_active":"2023-12-12T09:53:48.830363+0000","last_peered":"2023-12-12T09:53:48.830363+0000","last_clean":"2023-12-12T09:53:48.830363+0000","last_became_active":"2023-12-12T09:53:47.835000+0000","last_became_peered":"2023-12-12T09:53:47.835000+0000","last_unstale":"2023-12-12T09:53:48.830363+0000","last_undegraded":"2023-12-12T09:53:48.830363+0000","last_fullsized":"2023-12-12T09:53:48.830363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:53:42.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984158+0000","last_change":"2023-12-12T09:53:47.837046+0000","last_active":"2023-12-12T09:53:48.984158+0000","last_peered":"2023-12-12T09:53:48.984158+0000","last_clean":"2023-12-12T09:53:48.984158+0000","last_became_active":"2023-12-12T09:53:47.836708+0000","last_became_peered":"2023-12-12T09:53:47.836708+0000","last_unstale":"2023-12-12T09:53:48.984158+0000","last_undegraded":"2023-12-12T09:53:48.984158+0000","last_fullsized":"2023-12-12T09:53:48.984158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:03:35.377264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830037+0000","last_change":"2023-12-12T09:53:47.833134+0000","last_active":"2023-12-12T09:53:48.830037+0000","last_peered":"2023-12-12T09:53:48.830037+0000","last_clean":"2023-12-12T09:53:48.830037+0000","last_became_active":"2023-12-12T09:53:47.832842+0000","last_became_peered":"2023-12-12T09:53:47.832842+0000","last_unstale":"2023-12-12T09:53:48.830037+0000","last_undegraded":"2023-12-12T09:53:48.830037+0000","last_fullsized":"2023-12-12T09:53:48.830037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:11.328087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831022+0000","last_change":"2023-12-12T09:53:47.826921+0000","last_active":"2023-12-12T09:53:48.831022+0000","last_peered":"2023-12-12T09:53:48.831022+0000","last_clean":"2023-12-12T09:53:48.831022+0000","last_became_active":"2023-12-12T09:53:47.826625+0000","last_became_peered":"2023-12-12T09:53:47.826625+0000","last_unstale":"2023-12-12T09:53:48.831022+0000","last_undegraded":"2023-12-12T09:53:48.831022+0000","last_fullsized":"2023-12-12T09:53:48.831022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T09:57:14.932430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830661+0000","last_change":"2023-12-12T09:53:47.843667+0000","last_active":"2023-12-12T09:53:48.830661+0000","last_peered":"2023-12-12T09:53:48.830661+0000","last_clean":"2023-12-12T09:53:48.830661+0000","last_became_active":"2023-12-12T09:53:47.843178+0000","last_became_peered":"2023-12-12T09:53:47.843178+0000","last_unstale":"2023-12-12T09:53:48.830661+0000","last_undegraded":"2023-12-12T09:53:48.830661+0000","last_fullsized":"2023-12-12T09:53:48.830661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:18.340600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.840865+0000","last_change":"2023-12-12T09:53:47.831671+0000","last_active":"2023-12-12T09:53:47.840865+0000","last_peered":"2023-12-12T09:53:47.840865+0000","last_clean":"2023-12-12T09:53:47.840865+0000","last_became_active":"2023-12-12T09:53:47.831377+0000","last_became_peered":"2023-12-12T09:53:47.831377+0000","last_unstale":"2023-12-12T09:53:47.840865+0000","last_undegraded":"2023-12-12T09:53:47.840865+0000","last_fullsized":"2023-12-12T09:53:47.840865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:23:53.334470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830763+0000","last_change":"2023-12-12T09:53:47.843414+0000","last_active":"2023-12-12T09:53:48.830763+0000","last_peered":"2023-12-12T09:53:48.830763+0000","last_clean":"2023-12-12T09:53:48.830763+0000","last_became_active":"2023-12-12T09:53:47.843114+0000","last_became_peered":"2023-12-12T09:53:47.843114+0000","last_unstale":"2023-12-12T09:53:48.830763+0000","last_undegraded":"2023-12-12T09:53:48.830763+0000","last_fullsized":"2023-12-12T09:53:48.830763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:12.742806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830458+0000","last_change":"2023-12-12T09:53:46.831664+0000","last_active":"2023-12-12T09:53:48.830458+0000","last_peered":"2023-12-12T09:53:48.830458+0000","last_clean":"2023-12-12T09:53:48.830458+0000","last_became_active":"2023-12-12T09:53:46.831342+0000","last_became_peered":"2023-12-12T09:53:46.831342+0000","last_unstale":"2023-12-12T09:53:48.830458+0000","last_undegraded":"2023-12-12T09:53:48.830458+0000","last_fullsized":"2023-12-12T09:53:48.830458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:44:10.625305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.841767+0000","last_change":"2023-12-12T09:53:47.838090+0000","last_active":"2023-12-12T09:53:47.841767+0000","last_peered":"2023-12-12T09:53:47.841767+0000","last_clean":"2023-12-12T09:53:47.841767+0000","last_became_active":"2023-12-12T09:53:47.837705+0000","last_became_peered":"2023-12-12T09:53:47.837705+0000","last_unstale":"2023-12-12T09:53:47.841767+0000","last_undegraded":"2023-12-12T09:53:47.841767+0000","last_fullsized":"2023-12-12T09:53:47.841767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:00:50.227315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.859202+0000","last_change":"2023-12-12T09:53:46.819363+0000","last_active":"2023-12-12T09:53:47.859202+0000","last_peered":"2023-12-12T09:53:47.859202+0000","last_clean":"2023-12-12T09:53:47.859202+0000","last_became_active":"2023-12-12T09:53:46.819066+0000","last_became_peered":"2023-12-12T09:53:46.819066+0000","last_unstale":"2023-12-12T09:53:47.859202+0000","last_undegraded":"2023-12-12T09:53:47.859202+0000","last_fullsized":"2023-12-12T09:53:47.859202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:19.536305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188466+0000","last_change":"2023-12-12T09:53:47.841751+0000","last_active":"2023-12-12T09:53:49.188466+0000","last_peered":"2023-12-12T09:53:49.188466+0000","last_clean":"2023-12-12T09:53:49.188466+0000","last_became_active":"2023-12-12T09:53:47.839912+0000","last_became_peered":"2023-12-12T09:53:47.839912+0000","last_unstale":"2023-12-12T09:53:49.188466+0000","last_undegraded":"2023-12-12T09:53:49.188466+0000","last_fullsized":"2023-12-12T09:53:49.188466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:18.340600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.849655+0000","last_change":"2023-12-12T09:53:46.821382+0000","last_active":"2023-12-12T09:53:47.849655+0000","last_peered":"2023-12-12T09:53:47.849655+0000","last_clean":"2023-12-12T09:53:47.849655+0000","last_became_active":"2023-12-12T09:53:46.820805+0000","last_became_peered":"2023-12-12T09:53:46.820805+0000","last_unstale":"2023-12-12T09:53:47.849655+0000","last_undegraded":"2023-12-12T09:53:47.849655+0000","last_fullsized":"2023-12-12T09:53:47.849655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:19.536305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.831923+0000","last_change":"2023-12-12T09:53:47.824893+0000","last_active":"2023-12-12T09:53:47.831923+0000","last_peered":"2023-12-12T09:53:47.831923+0000","last_clean":"2023-12-12T09:53:47.831923+0000","last_became_active":"2023-12-12T09:53:47.824455+0000","last_became_peered":"2023-12-12T09:53:47.824455+0000","last_unstale":"2023-12-12T09:53:47.831923+0000","last_undegraded":"2023-12-12T09:53:47.831923+0000","last_fullsized":"2023-12-12T09:53:47.831923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:21:23.166634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188176+0000","last_change":"2023-12-12T09:53:46.824902+0000","last_active":"2023-12-12T09:53:49.188176+0000","last_peered":"2023-12-12T09:53:49.188176+0000","last_clean":"2023-12-12T09:53:49.188176+0000","last_became_active":"2023-12-12T09:53:46.820316+0000","last_became_peered":"2023-12-12T09:53:46.820316+0000","last_unstale":"2023-12-12T09:53:49.188176+0000","last_undegraded":"2023-12-12T09:53:49.188176+0000","last_fullsized":"2023-12-12T09:53:49.188176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:11:17.544738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.848858+0000","last_change":"2023-12-12T09:53:47.840807+0000","last_active":"2023-12-12T09:53:47.848858+0000","last_peered":"2023-12-12T09:53:47.848858+0000","last_clean":"2023-12-12T09:53:47.848858+0000","last_became_active":"2023-12-12T09:53:47.834899+0000","last_became_peered":"2023-12-12T09:53:47.834899+0000","last_unstale":"2023-12-12T09:53:47.848858+0000","last_undegraded":"2023-12-12T09:53:47.848858+0000","last_fullsized":"2023-12-12T09:53:47.848858+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:14:46.641653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983952+0000","last_change":"2023-12-12T09:53:46.821849+0000","last_active":"2023-12-12T09:53:48.983952+0000","last_peered":"2023-12-12T09:53:48.983952+0000","last_clean":"2023-12-12T09:53:48.983952+0000","last_became_active":"2023-12-12T09:53:46.821522+0000","last_became_peered":"2023-12-12T09:53:46.821522+0000","last_unstale":"2023-12-12T09:53:48.983952+0000","last_undegraded":"2023-12-12T09:53:48.983952+0000","last_fullsized":"2023-12-12T09:53:48.983952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:23:38.111445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983891+0000","last_change":"2023-12-12T09:53:47.837124+0000","last_active":"2023-12-12T09:53:48.983891+0000","last_peered":"2023-12-12T09:53:48.983891+0000","last_clean":"2023-12-12T09:53:48.983891+0000","last_became_active":"2023-12-12T09:53:47.836775+0000","last_became_peered":"2023-12-12T09:53:47.836775+0000","last_unstale":"2023-12-12T09:53:48.983891+0000","last_undegraded":"2023-12-12T09:53:48.983891+0000","last_fullsized":"2023-12-12T09:53:48.983891+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:14.541246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830518+0000","last_change":"2023-12-12T09:53:46.832964+0000","last_active":"2023-12-12T09:53:48.830518+0000","last_peered":"2023-12-12T09:53:48.830518+0000","last_clean":"2023-12-12T09:53:48.830518+0000","last_became_active":"2023-12-12T09:53:46.830772+0000","last_became_peered":"2023-12-12T09:53:46.830772+0000","last_unstale":"2023-12-12T09:53:48.830518+0000","last_undegraded":"2023-12-12T09:53:48.830518+0000","last_fullsized":"2023-12-12T09:53:48.830518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:10.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188240+0000","last_change":"2023-12-12T09:53:47.840670+0000","last_active":"2023-12-12T09:53:49.188240+0000","last_peered":"2023-12-12T09:53:49.188240+0000","last_clean":"2023-12-12T09:53:49.188240+0000","last_became_active":"2023-12-12T09:53:47.837955+0000","last_became_peered":"2023-12-12T09:53:47.837955+0000","last_unstale":"2023-12-12T09:53:49.188240+0000","last_undegraded":"2023-12-12T09:53:49.188240+0000","last_fullsized":"2023-12-12T09:53:49.188240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:28.550271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.816114+0000","last_change":"2023-12-12T09:53:46.828799+0000","last_active":"2023-12-12T09:53:47.816114+0000","last_peered":"2023-12-12T09:53:47.816114+0000","last_clean":"2023-12-12T09:53:47.816114+0000","last_became_active":"2023-12-12T09:53:46.828519+0000","last_became_peered":"2023-12-12T09:53:46.828519+0000","last_unstale":"2023-12-12T09:53:47.816114+0000","last_undegraded":"2023-12-12T09:53:47.816114+0000","last_fullsized":"2023-12-12T09:53:47.816114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:11:17.544738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.839505+0000","last_change":"2023-12-12T09:53:47.835361+0000","last_active":"2023-12-12T09:53:47.839505+0000","last_peered":"2023-12-12T09:53:47.839505+0000","last_clean":"2023-12-12T09:53:47.839505+0000","last_became_active":"2023-12-12T09:53:47.835076+0000","last_became_peered":"2023-12-12T09:53:47.835076+0000","last_unstale":"2023-12-12T09:53:47.839505+0000","last_undegraded":"2023-12-12T09:53:47.839505+0000","last_fullsized":"2023-12-12T09:53:47.839505+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:53:42.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.187405+0000","last_change":"2023-12-12T09:53:46.833689+0000","last_active":"2023-12-12T09:53:49.187405+0000","last_peered":"2023-12-12T09:53:49.187405+0000","last_clean":"2023-12-12T09:53:49.187405+0000","last_became_active":"2023-12-12T09:53:46.833289+0000","last_became_peered":"2023-12-12T09:53:46.833289+0000","last_unstale":"2023-12-12T09:53:49.187405+0000","last_undegraded":"2023-12-12T09:53:49.187405+0000","last_fullsized":"2023-12-12T09:53:49.187405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:43:47.743822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978520+0000","last_change":"2023-12-12T09:53:47.838254+0000","last_active":"2023-12-12T09:53:48.978520+0000","last_peered":"2023-12-12T09:53:48.978520+0000","last_clean":"2023-12-12T09:53:48.978520+0000","last_became_active":"2023-12-12T09:53:47.833762+0000","last_became_peered":"2023-12-12T09:53:47.833762+0000","last_unstale":"2023-12-12T09:53:48.978520+0000","last_undegraded":"2023-12-12T09:53:48.978520+0000","last_fullsized":"2023-12-12T09:53:48.978520+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:52:38.152266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.807661+0000","last_change":"2023-12-12T09:53:46.815733+0000","last_active":"2023-12-12T09:53:47.807661+0000","last_peered":"2023-12-12T09:53:47.807661+0000","last_clean":"2023-12-12T09:53:47.807661+0000","last_became_active":"2023-12-12T09:53:46.815440+0000","last_became_peered":"2023-12-12T09:53:46.815440+0000","last_unstale":"2023-12-12T09:53:47.807661+0000","last_undegraded":"2023-12-12T09:53:47.807661+0000","last_fullsized":"2023-12-12T09:53:47.807661+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:46:24.806438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830939+0000","last_change":"2023-12-12T09:53:47.848383+0000","last_active":"2023-12-12T09:53:48.830939+0000","last_peered":"2023-12-12T09:53:48.830939+0000","last_clean":"2023-12-12T09:53:48.830939+0000","last_became_active":"2023-12-12T09:53:47.846130+0000","last_became_peered":"2023-12-12T09:53:47.846130+0000","last_unstale":"2023-12-12T09:53:48.830939+0000","last_undegraded":"2023-12-12T09:53:48.830939+0000","last_fullsized":"2023-12-12T09:53:48.830939+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:33.797186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984867+0000","last_change":"2023-12-12T09:53:46.828640+0000","last_active":"2023-12-12T09:53:48.984867+0000","last_peered":"2023-12-12T09:53:48.984867+0000","last_clean":"2023-12-12T09:53:48.984867+0000","last_became_active":"2023-12-12T09:53:46.828353+0000","last_became_peered":"2023-12-12T09:53:46.828353+0000","last_unstale":"2023-12-12T09:53:48.984867+0000","last_undegraded":"2023-12-12T09:53:48.984867+0000","last_fullsized":"2023-12-12T09:53:48.984867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:48:00.098770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.851037+0000","last_change":"2023-12-12T09:53:47.845944+0000","last_active":"2023-12-12T09:53:47.851037+0000","last_peered":"2023-12-12T09:53:47.851037+0000","last_clean":"2023-12-12T09:53:47.851037+0000","last_became_active":"2023-12-12T09:53:47.845599+0000","last_became_peered":"2023-12-12T09:53:47.845599+0000","last_unstale":"2023-12-12T09:53:47.851037+0000","last_undegraded":"2023-12-12T09:53:47.851037+0000","last_fullsized":"2023-12-12T09:53:47.851037+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:28.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984626+0000","last_change":"2023-12-12T09:53:46.826917+0000","last_active":"2023-12-12T09:53:48.984626+0000","last_peered":"2023-12-12T09:53:48.984626+0000","last_clean":"2023-12-12T09:53:48.984626+0000","last_became_active":"2023-12-12T09:53:46.826642+0000","last_became_peered":"2023-12-12T09:53:46.826642+0000","last_unstale":"2023-12-12T09:53:48.984626+0000","last_undegraded":"2023-12-12T09:53:48.984626+0000","last_fullsized":"2023-12-12T09:53:48.984626+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:17.328205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188620+0000","last_change":"2023-12-12T09:53:47.830875+0000","last_active":"2023-12-12T09:53:49.188620+0000","last_peered":"2023-12-12T09:53:49.188620+0000","last_clean":"2023-12-12T09:53:49.188620+0000","last_became_active":"2023-12-12T09:53:47.829956+0000","last_became_peered":"2023-12-12T09:53:47.829956+0000","last_unstale":"2023-12-12T09:53:49.188620+0000","last_undegraded":"2023-12-12T09:53:49.188620+0000","last_fullsized":"2023-12-12T09:53:49.188620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:48:01.111165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.860872+0000","last_change":"2023-12-12T09:53:46.818760+0000","last_active":"2023-12-12T09:53:47.860872+0000","last_peered":"2023-12-12T09:53:47.860872+0000","last_clean":"2023-12-12T09:53:47.860872+0000","last_became_active":"2023-12-12T09:53:46.818343+0000","last_became_peered":"2023-12-12T09:53:46.818343+0000","last_unstale":"2023-12-12T09:53:47.860872+0000","last_undegraded":"2023-12-12T09:53:47.860872+0000","last_fullsized":"2023-12-12T09:53:47.860872+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:23:38.111445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.848877+0000","last_change":"2023-12-12T09:53:47.844706+0000","last_active":"2023-12-12T09:53:47.848877+0000","last_peered":"2023-12-12T09:53:47.848877+0000","last_clean":"2023-12-12T09:53:47.848877+0000","last_became_active":"2023-12-12T09:53:47.844332+0000","last_became_peered":"2023-12-12T09:53:47.844332+0000","last_unstale":"2023-12-12T09:53:47.848877+0000","last_undegraded":"2023-12-12T09:53:47.848877+0000","last_fullsized":"2023-12-12T09:53:47.848877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:10:19.295434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.851457+0000","last_change":"2023-12-12T09:53:46.812322+0000","last_active":"2023-12-12T09:53:47.851457+0000","last_peered":"2023-12-12T09:53:47.851457+0000","last_clean":"2023-12-12T09:53:47.851457+0000","last_became_active":"2023-12-12T09:53:46.812053+0000","last_became_peered":"2023-12-12T09:53:46.812053+0000","last_unstale":"2023-12-12T09:53:47.851457+0000","last_undegraded":"2023-12-12T09:53:47.851457+0000","last_fullsized":"2023-12-12T09:53:47.851457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:55:01.516430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.839866+0000","last_change":"2023-12-12T09:53:47.832667+0000","last_active":"2023-12-12T09:53:47.839866+0000","last_peered":"2023-12-12T09:53:47.839866+0000","last_clean":"2023-12-12T09:53:47.839866+0000","last_became_active":"2023-12-12T09:53:47.832364+0000","last_became_peered":"2023-12-12T09:53:47.832364+0000","last_unstale":"2023-12-12T09:53:47.839866+0000","last_undegraded":"2023-12-12T09:53:47.839866+0000","last_fullsized":"2023-12-12T09:53:47.839866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:28.550271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.816297+0000","last_change":"2023-12-12T09:53:46.828674+0000","last_active":"2023-12-12T09:53:47.816297+0000","last_peered":"2023-12-12T09:53:47.816297+0000","last_clean":"2023-12-12T09:53:47.816297+0000","last_became_active":"2023-12-12T09:53:46.828409+0000","last_became_peered":"2023-12-12T09:53:46.828409+0000","last_unstale":"2023-12-12T09:53:47.816297+0000","last_undegraded":"2023-12-12T09:53:47.816297+0000","last_fullsized":"2023-12-12T09:53:47.816297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:14:45.629258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830851+0000","last_change":"2023-12-12T09:53:47.838530+0000","last_active":"2023-12-12T09:53:48.830851+0000","last_peered":"2023-12-12T09:53:48.830851+0000","last_clean":"2023-12-12T09:53:48.830851+0000","last_became_active":"2023-12-12T09:53:47.838219+0000","last_became_peered":"2023-12-12T09:53:47.838219+0000","last_unstale":"2023-12-12T09:53:48.830851+0000","last_undegraded":"2023-12-12T09:53:48.830851+0000","last_fullsized":"2023-12-12T09:53:48.830851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:55:02.528825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.829616+0000","last_change":"2023-12-12T09:53:46.830449+0000","last_active":"2023-12-12T09:53:48.829616+0000","last_peered":"2023-12-12T09:53:48.829616+0000","last_clean":"2023-12-12T09:53:48.829616+0000","last_became_active":"2023-12-12T09:53:46.830174+0000","last_became_peered":"2023-12-12T09:53:46.830174+0000","last_unstale":"2023-12-12T09:53:48.829616+0000","last_undegraded":"2023-12-12T09:53:48.829616+0000","last_fullsized":"2023-12-12T09:53:48.829616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:08:40.958883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.844239+0000","last_change":"2023-12-12T09:53:47.837719+0000","last_active":"2023-12-12T09:53:47.844239+0000","last_peered":"2023-12-12T09:53:47.844239+0000","last_clean":"2023-12-12T09:53:47.844239+0000","last_became_active":"2023-12-12T09:53:47.837436+0000","last_became_peered":"2023-12-12T09:53:47.837436+0000","last_unstale":"2023-12-12T09:53:47.844239+0000","last_undegraded":"2023-12-12T09:53:47.844239+0000","last_fullsized":"2023-12-12T09:53:47.844239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:48:01.111165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.829529+0000","last_change":"2023-12-12T09:53:46.831988+0000","last_active":"2023-12-12T09:53:48.829529+0000","last_peered":"2023-12-12T09:53:48.829529+0000","last_clean":"2023-12-12T09:53:48.829529+0000","last_became_active":"2023-12-12T09:53:46.831720+0000","last_became_peered":"2023-12-12T09:53:46.831720+0000","last_unstale":"2023-12-12T09:53:48.829529+0000","last_undegraded":"2023-12-12T09:53:48.829529+0000","last_fullsized":"2023-12-12T09:53:48.829529+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:00:49.214920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.832374+0000","last_change":"2023-12-12T09:53:47.821486+0000","last_active":"2023-12-12T09:53:47.832374+0000","last_peered":"2023-12-12T09:53:47.832374+0000","last_clean":"2023-12-12T09:53:47.832374+0000","last_became_active":"2023-12-12T09:53:47.821184+0000","last_became_peered":"2023-12-12T09:53:47.821184+0000","last_unstale":"2023-12-12T09:53:47.832374+0000","last_undegraded":"2023-12-12T09:53:47.832374+0000","last_fullsized":"2023-12-12T09:53:47.832374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:18.340600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984845+0000","last_change":"2023-12-12T09:53:46.832373+0000","last_active":"2023-12-12T09:53:48.984845+0000","last_peered":"2023-12-12T09:53:48.984845+0000","last_clean":"2023-12-12T09:53:48.984845+0000","last_became_active":"2023-12-12T09:53:46.830497+0000","last_became_peered":"2023-12-12T09:53:46.830497+0000","last_unstale":"2023-12-12T09:53:48.984845+0000","last_undegraded":"2023-12-12T09:53:48.984845+0000","last_fullsized":"2023-12-12T09:53:48.984845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:25.240536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984442+0000","last_change":"2023-12-12T09:53:47.834881+0000","last_active":"2023-12-12T09:53:48.984442+0000","last_peered":"2023-12-12T09:53:48.984442+0000","last_clean":"2023-12-12T09:53:48.984442+0000","last_became_active":"2023-12-12T09:53:47.834078+0000","last_became_peered":"2023-12-12T09:53:47.834078+0000","last_unstale":"2023-12-12T09:53:48.984442+0000","last_undegraded":"2023-12-12T09:53:48.984442+0000","last_fullsized":"2023-12-12T09:53:48.984442+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:34:42.332004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188478+0000","last_change":"2023-12-12T09:53:46.831524+0000","last_active":"2023-12-12T09:53:49.188478+0000","last_peered":"2023-12-12T09:53:49.188478+0000","last_clean":"2023-12-12T09:53:49.188478+0000","last_became_active":"2023-12-12T09:53:46.830628+0000","last_became_peered":"2023-12-12T09:53:46.830628+0000","last_unstale":"2023-12-12T09:53:49.188478+0000","last_undegraded":"2023-12-12T09:53:49.188478+0000","last_fullsized":"2023-12-12T09:53:49.188478+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:19.536305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.188330+0000","last_change":"2023-12-12T09:53:47.833334+0000","last_active":"2023-12-12T09:53:49.188330+0000","last_peered":"2023-12-12T09:53:49.188330+0000","last_clean":"2023-12-12T09:53:49.188330+0000","last_became_active":"2023-12-12T09:53:47.832994+0000","last_became_peered":"2023-12-12T09:53:47.832994+0000","last_unstale":"2023-12-12T09:53:49.188330+0000","last_undegraded":"2023-12-12T09:53:49.188330+0000","last_fullsized":"2023-12-12T09:53:49.188330+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:23:39.123840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978526+0000","last_change":"2023-12-12T09:53:46.822712+0000","last_active":"2023-12-12T09:53:48.978526+0000","last_peered":"2023-12-12T09:53:48.978526+0000","last_clean":"2023-12-12T09:53:48.978526+0000","last_became_active":"2023-12-12T09:53:46.821700+0000","last_became_peered":"2023-12-12T09:53:46.821700+0000","last_unstale":"2023-12-12T09:53:48.978526+0000","last_undegraded":"2023-12-12T09:53:48.978526+0000","last_fullsized":"2023-12-12T09:53:48.978526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:00:47.182940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978811+0000","last_change":"2023-12-12T09:53:47.834751+0000","last_active":"2023-12-12T09:53:48.978811+0000","last_peered":"2023-12-12T09:53:48.978811+0000","last_clean":"2023-12-12T09:53:48.978811+0000","last_became_active":"2023-12-12T09:53:47.834435+0000","last_became_peered":"2023-12-12T09:53:47.834435+0000","last_unstale":"2023-12-12T09:53:48.978811+0000","last_undegraded":"2023-12-12T09:53:48.978811+0000","last_fullsized":"2023-12-12T09:53:48.978811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:29:49.006050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.816472+0000","last_change":"2023-12-12T09:53:46.821077+0000","last_active":"2023-12-12T09:53:47.816472+0000","last_peered":"2023-12-12T09:53:47.816472+0000","last_clean":"2023-12-12T09:53:47.816472+0000","last_became_active":"2023-12-12T09:53:46.820774+0000","last_became_peered":"2023-12-12T09:53:46.820774+0000","last_unstale":"2023-12-12T09:53:47.816472+0000","last_undegraded":"2023-12-12T09:53:47.816472+0000","last_fullsized":"2023-12-12T09:53:47.816472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:27.790822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831798+0000","last_change":"2023-12-12T09:53:47.838347+0000","last_active":"2023-12-12T09:53:48.831798+0000","last_peered":"2023-12-12T09:53:48.831798+0000","last_clean":"2023-12-12T09:53:48.831798+0000","last_became_active":"2023-12-12T09:53:47.837877+0000","last_became_peered":"2023-12-12T09:53:47.837877+0000","last_unstale":"2023-12-12T09:53:48.831798+0000","last_undegraded":"2023-12-12T09:53:48.831798+0000","last_fullsized":"2023-12-12T09:53:48.831798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:14.541246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978294+0000","last_change":"2023-12-12T09:53:46.814925+0000","last_active":"2023-12-12T09:53:48.978294+0000","last_peered":"2023-12-12T09:53:48.978294+0000","last_clean":"2023-12-12T09:53:48.978294+0000","last_became_active":"2023-12-12T09:53:46.814337+0000","last_became_peered":"2023-12-12T09:53:46.814337+0000","last_unstale":"2023-12-12T09:53:48.978294+0000","last_undegraded":"2023-12-12T09:53:48.978294+0000","last_fullsized":"2023-12-12T09:53:48.978294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:54:56.189089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.848805+0000","last_change":"2023-12-12T09:53:47.838780+0000","last_active":"2023-12-12T09:53:47.848805+0000","last_peered":"2023-12-12T09:53:47.848805+0000","last_clean":"2023-12-12T09:53:47.848805+0000","last_became_active":"2023-12-12T09:53:47.838516+0000","last_became_peered":"2023-12-12T09:53:47.838516+0000","last_unstale":"2023-12-12T09:53:47.848805+0000","last_undegraded":"2023-12-12T09:53:47.848805+0000","last_fullsized":"2023-12-12T09:53:47.848805+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:44:11.637699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978457+0000","last_change":"2023-12-12T09:53:46.823815+0000","last_active":"2023-12-12T09:53:48.978457+0000","last_peered":"2023-12-12T09:53:48.978457+0000","last_clean":"2023-12-12T09:53:48.978457+0000","last_became_active":"2023-12-12T09:53:46.823326+0000","last_became_peered":"2023-12-12T09:53:46.823326+0000","last_unstale":"2023-12-12T09:53:48.978457+0000","last_undegraded":"2023-12-12T09:53:48.978457+0000","last_fullsized":"2023-12-12T09:53:48.978457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:23:42.082271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.983494+0000","last_change":"2023-12-12T09:53:47.825003+0000","last_active":"2023-12-12T09:53:48.983494+0000","last_peered":"2023-12-12T09:53:48.983494+0000","last_clean":"2023-12-12T09:53:48.983494+0000","last_became_active":"2023-12-12T09:53:47.824524+0000","last_became_peered":"2023-12-12T09:53:47.824524+0000","last_unstale":"2023-12-12T09:53:48.983494+0000","last_undegraded":"2023-12-12T09:53:48.983494+0000","last_fullsized":"2023-12-12T09:53:48.983494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:21:23.166634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830881+0000","last_change":"2023-12-12T09:53:46.834084+0000","last_active":"2023-12-12T09:53:48.830881+0000","last_peered":"2023-12-12T09:53:48.830881+0000","last_clean":"2023-12-12T09:53:48.830881+0000","last_became_active":"2023-12-12T09:53:46.833773+0000","last_became_peered":"2023-12-12T09:53:46.833773+0000","last_unstale":"2023-12-12T09:53:48.830881+0000","last_undegraded":"2023-12-12T09:53:48.830881+0000","last_fullsized":"2023-12-12T09:53:48.830881+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:44:10.625305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.839485+0000","last_change":"2023-12-12T09:53:47.835284+0000","last_active":"2023-12-12T09:53:47.839485+0000","last_peered":"2023-12-12T09:53:47.839485+0000","last_clean":"2023-12-12T09:53:47.839485+0000","last_became_active":"2023-12-12T09:53:47.834989+0000","last_became_peered":"2023-12-12T09:53:47.834989+0000","last_unstale":"2023-12-12T09:53:47.839485+0000","last_undegraded":"2023-12-12T09:53:47.839485+0000","last_fullsized":"2023-12-12T09:53:47.839485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:12.742806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.978348+0000","last_change":"2023-12-12T09:53:46.812854+0000","last_active":"2023-12-12T09:53:48.978348+0000","last_peered":"2023-12-12T09:53:48.978348+0000","last_clean":"2023-12-12T09:53:48.978348+0000","last_became_active":"2023-12-12T09:53:46.812504+0000","last_became_peered":"2023-12-12T09:53:46.812504+0000","last_unstale":"2023-12-12T09:53:48.978348+0000","last_undegraded":"2023-12-12T09:53:48.978348+0000","last_fullsized":"2023-12-12T09:53:48.978348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:34:12.676764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.837714+0000","last_change":"2023-12-12T09:53:47.830923+0000","last_active":"2023-12-12T09:53:47.837714+0000","last_peered":"2023-12-12T09:53:47.837714+0000","last_clean":"2023-12-12T09:53:47.837714+0000","last_became_active":"2023-12-12T09:53:47.830631+0000","last_became_peered":"2023-12-12T09:53:47.830631+0000","last_unstale":"2023-12-12T09:53:47.837714+0000","last_undegraded":"2023-12-12T09:53:47.837714+0000","last_fullsized":"2023-12-12T09:53:47.837714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:10:19.295434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.985015+0000","last_change":"2023-12-12T09:53:46.833010+0000","last_active":"2023-12-12T09:53:48.985015+0000","last_peered":"2023-12-12T09:53:48.985015+0000","last_clean":"2023-12-12T09:53:48.985015+0000","last_became_active":"2023-12-12T09:53:46.829781+0000","last_became_peered":"2023-12-12T09:53:46.829781+0000","last_unstale":"2023-12-12T09:53:48.985015+0000","last_undegraded":"2023-12-12T09:53:48.985015+0000","last_fullsized":"2023-12-12T09:53:48.985015+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:11.730411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831370+0000","last_change":"2023-12-12T09:53:47.851831+0000","last_active":"2023-12-12T09:53:48.831370+0000","last_peered":"2023-12-12T09:53:48.831370+0000","last_clean":"2023-12-12T09:53:48.831370+0000","last_became_active":"2023-12-12T09:53:47.846500+0000","last_became_peered":"2023-12-12T09:53:47.846500+0000","last_unstale":"2023-12-12T09:53:48.831370+0000","last_undegraded":"2023-12-12T09:53:48.831370+0000","last_fullsized":"2023-12-12T09:53:48.831370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:35:29.440308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.981818+0000","last_change":"2023-12-12T09:53:46.827438+0000","last_active":"2023-12-12T09:53:48.981818+0000","last_peered":"2023-12-12T09:53:48.981818+0000","last_clean":"2023-12-12T09:53:48.981818+0000","last_became_active":"2023-12-12T09:53:46.827161+0000","last_became_peered":"2023-12-12T09:53:46.827161+0000","last_unstale":"2023-12-12T09:53:48.981818+0000","last_undegraded":"2023-12-12T09:53:48.981818+0000","last_fullsized":"2023-12-12T09:53:48.981818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:24:53.309495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.837272+0000","last_change":"2023-12-12T09:53:47.831845+0000","last_active":"2023-12-12T09:53:47.837272+0000","last_peered":"2023-12-12T09:53:47.837272+0000","last_clean":"2023-12-12T09:53:47.837272+0000","last_became_active":"2023-12-12T09:53:47.831516+0000","last_became_peered":"2023-12-12T09:53:47.831516+0000","last_unstale":"2023-12-12T09:53:47.837272+0000","last_undegraded":"2023-12-12T09:53:47.837272+0000","last_fullsized":"2023-12-12T09:53:47.837272+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:33.797186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.821902+0000","last_change":"2023-12-12T09:53:46.813712+0000","last_active":"2023-12-12T09:53:47.821902+0000","last_peered":"2023-12-12T09:53:47.821902+0000","last_clean":"2023-12-12T09:53:47.821902+0000","last_became_active":"2023-12-12T09:53:46.813410+0000","last_became_peered":"2023-12-12T09:53:46.813410+0000","last_unstale":"2023-12-12T09:53:47.821902+0000","last_undegraded":"2023-12-12T09:53:47.821902+0000","last_fullsized":"2023-12-12T09:53:47.821902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:11:17.544738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.848840+0000","last_change":"2023-12-12T09:53:47.835507+0000","last_active":"2023-12-12T09:53:47.848840+0000","last_peered":"2023-12-12T09:53:47.848840+0000","last_clean":"2023-12-12T09:53:47.848840+0000","last_became_active":"2023-12-12T09:53:47.834559+0000","last_became_peered":"2023-12-12T09:53:47.834559+0000","last_unstale":"2023-12-12T09:53:47.848840+0000","last_undegraded":"2023-12-12T09:53:47.848840+0000","last_fullsized":"2023-12-12T09:53:47.848840+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:05.777453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.825161+0000","last_change":"2023-12-12T09:53:46.830561+0000","last_active":"2023-12-12T09:53:47.825161+0000","last_peered":"2023-12-12T09:53:47.825161+0000","last_clean":"2023-12-12T09:53:47.825161+0000","last_became_active":"2023-12-12T09:53:46.830224+0000","last_became_peered":"2023-12-12T09:53:46.830224+0000","last_unstale":"2023-12-12T09:53:47.825161+0000","last_undegraded":"2023-12-12T09:53:47.825161+0000","last_fullsized":"2023-12-12T09:53:47.825161+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.834414+0000","last_change":"2023-12-12T09:53:47.829411+0000","last_active":"2023-12-12T09:53:47.834414+0000","last_peered":"2023-12-12T09:53:47.834414+0000","last_clean":"2023-12-12T09:53:47.834414+0000","last_became_active":"2023-12-12T09:53:47.829087+0000","last_became_peered":"2023-12-12T09:53:47.829087+0000","last_unstale":"2023-12-12T09:53:47.834414+0000","last_undegraded":"2023-12-12T09:53:47.834414+0000","last_fullsized":"2023-12-12T09:53:47.834414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:08:41.971277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831243+0000","last_change":"2023-12-12T09:53:46.824362+0000","last_active":"2023-12-12T09:53:48.831243+0000","last_peered":"2023-12-12T09:53:48.831243+0000","last_clean":"2023-12-12T09:53:48.831243+0000","last_became_active":"2023-12-12T09:53:46.824009+0000","last_became_peered":"2023-12-12T09:53:46.824009+0000","last_unstale":"2023-12-12T09:53:48.831243+0000","last_undegraded":"2023-12-12T09:53:48.831243+0000","last_fullsized":"2023-12-12T09:53:48.831243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:10:18.283040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.187614+0000","last_change":"2023-12-12T09:53:47.831689+0000","last_active":"2023-12-12T09:53:49.187614+0000","last_peered":"2023-12-12T09:53:49.187614+0000","last_clean":"2023-12-12T09:53:49.187614+0000","last_became_active":"2023-12-12T09:53:47.828381+0000","last_became_peered":"2023-12-12T09:53:47.828381+0000","last_unstale":"2023-12-12T09:53:49.187614+0000","last_undegraded":"2023-12-12T09:53:49.187614+0000","last_fullsized":"2023-12-12T09:53:49.187614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:53:42.806829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:49.187405+0000","last_change":"2023-12-12T09:53:46.824875+0000","last_active":"2023-12-12T09:53:49.187405+0000","last_peered":"2023-12-12T09:53:49.187405+0000","last_clean":"2023-12-12T09:53:49.187405+0000","last_became_active":"2023-12-12T09:53:46.824589+0000","last_became_peered":"2023-12-12T09:53:46.824589+0000","last_unstale":"2023-12-12T09:53:49.187405+0000","last_undegraded":"2023-12-12T09:53:49.187405+0000","last_fullsized":"2023-12-12T09:53:49.187405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:42:04.765059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.831736+0000","last_change":"2023-12-12T09:53:47.847317+0000","last_active":"2023-12-12T09:53:48.831736+0000","last_peered":"2023-12-12T09:53:48.831736+0000","last_clean":"2023-12-12T09:53:48.831736+0000","last_became_active":"2023-12-12T09:53:47.846186+0000","last_became_peered":"2023-12-12T09:53:47.846186+0000","last_unstale":"2023-12-12T09:53:48.831736+0000","last_undegraded":"2023-12-12T09:53:48.831736+0000","last_fullsized":"2023-12-12T09:53:48.831736+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:46:25.818833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.802047+0000","last_change":"2023-12-12T09:53:46.814276+0000","last_active":"2023-12-12T09:53:47.802047+0000","last_peered":"2023-12-12T09:53:47.802047+0000","last_clean":"2023-12-12T09:53:47.802047+0000","last_became_active":"2023-12-12T09:53:46.813841+0000","last_became_peered":"2023-12-12T09:53:46.813841+0000","last_unstale":"2023-12-12T09:53:47.802047+0000","last_undegraded":"2023-12-12T09:53:47.802047+0000","last_fullsized":"2023-12-12T09:53:47.802047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.984737+0000","last_change":"2023-12-12T09:53:47.833754+0000","last_active":"2023-12-12T09:53:48.984737+0000","last_peered":"2023-12-12T09:53:48.984737+0000","last_clean":"2023-12-12T09:53:48.984737+0000","last_became_active":"2023-12-12T09:53:47.833460+0000","last_became_peered":"2023-12-12T09:53:47.833460+0000","last_unstale":"2023-12-12T09:53:48.984737+0000","last_undegraded":"2023-12-12T09:53:48.984737+0000","last_fullsized":"2023-12-12T09:53:48.984737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:46.777655+0000","last_clean_scrub_stamp":"2023-12-12T09:53:46.777655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:24:55.396686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-12T09:53:48.830219+0000","last_change":"2023-12-12T09:53:46.826607+0000","last_active":"2023-12-12T09:53:48.830219+0000","last_peered":"2023-12-12T09:53:48.830219+0000","last_clean":"2023-12-12T09:53:48.830219+0000","last_became_active":"2023-12-12T09:53:46.826318+0000","last_became_peered":"2023-12-12T09:53:46.826318+0000","last_unstale":"2023-12-12T09:53:48.830219+0000","last_undegraded":"2023-12-12T09:53:48.830219+0000","last_fullsized":"2023-12-12T09:53:48.830219+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:53:45.765260+0000","last_clean_scrub_stamp":"2023-12-12T09:53:45.765260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:15:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-12T09:53:47.821686+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:53:47.821686+0000","last_peered":"2023-12-12T09:53:47.821686+0000","last_clean":"2023-12-12T09:53:47.821686+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:53:47.821686+0000","last_undegraded":"2023-12-12T09:53:47.821686+0000","last_fullsized":"2023-12-12T09:53:47.821686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":780550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.85899999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.5509999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.306},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":6.6210000000000004,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.0129999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":1.619},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.371}]},{"osd":4,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.518}]},{"osd":5,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.585,"15min":1.585},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.5979999999999999,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.452}]},{"osd":6,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.633,"15min":1.633},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.044},{"interface":"front","average":{"1min":1.641,"5min":1.641,"15min":1.641},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.8300000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.4139999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":189082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.04},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.86899999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.381}]},{"osd":2,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.597}]},{"osd":3,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.4419999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":4.3209999999999997,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.5720000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.097},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":1.069,"5min":1.069,"15min":1.069},"max":{"1min":3.1989999999999998,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.1379999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":188612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":0.89200000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":3.8260000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.165},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.274},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.3100000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":3.944,"5min":3.944,"15min":3.944},"last":1.363},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.097}]},{"osd":4,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.462}]},{"osd":6,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.413},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.6060000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.1500000000000004,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.673,"15min":1.673},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.4950000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.79},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.96099999999999997}]},{"osd":1,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.079},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.6060000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.325},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.2629999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.47},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.167}]},{"osd":5,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"last":1.6899999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.7330000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.5229999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:53:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":1.863},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.74}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":183526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.627}]},{"osd":7,"last update":"Tue Dec 12 09:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.59499999999999997}]},{"osd":1,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.022}]},{"osd":3,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.95799999999999996}]},{"osd":4,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.482},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.2589999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.645}]},{"osd":6,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.3480000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.8130000000000002,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.599}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.876}]},{"osd":2,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.81499999999999995}]},{"osd":3,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.70999999999999996}]},{"osd":4,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":1.044}]},{"osd":5,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.95699999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":3.9209999999999998,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.0089999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":3.052,"5min":3.052,"15min":3.052},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.1369999999999996,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":1.222}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":771263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.0819999999999999,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":1.3420000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.8690000000000002,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":3.9940000000000002,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":1.137}]},{"osd":3,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.9049999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.012}]},{"osd":4,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.577},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":1.4790000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.597},{"interface":"front","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.54,"5min":2.54,"15min":2.54},"last":1.5109999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":1.5409999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:53:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":4.548,"5min":4.548,"15min":4.548},"last":1.385}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:53:52.770 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-12T09:53:52.770 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-12T09:53:53.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.032+0000 7fc58d5bb640 1 Processor -- start 2023-12-12T09:53:53.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.032+0000 7fc58d5bb640 1 -- start start 2023-12-12T09:53:53.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.034+0000 7fc58d5bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58815d240 0x7fc588070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.034+0000 7fc58d5bb640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc588071800 con 0x7fc5880cf8b0 2023-12-12T09:53:53.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.034+0000 7fc58d5bb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58815d620 con 0x7fc5880ac860 2023-12-12T09:53:53.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.034+0000 7fc58d5bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc588070f60 con 0x7fc58815d240 2023-12-12T09:53:53.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.034+0000 7fc5877fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc5880cf8b0 0x7fc5880b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59968/0 (socket says 172.21.15.72:59968) 2023-12-12T09:53:53.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.034+0000 7fc5877fe640 1 -- 172.21.15.72:0/3164376312 learned_addr learned my addr 172.21.15.72:0/3164376312 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:53.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.034+0000 7fc586ffd640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58815d240 0x7fc588070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031304268 0 0) 0x7fc588071800 con 0x7fc5880cf8b0 2023-12-12T09:53:53.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc570003520 con 0x7fc5880cf8b0 2023-12-12T09:53:53.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4076399037 0 0) 0x7fc570003520 con 0x7fc5880cf8b0 2023-12-12T09:53:53.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58815d240 msgr2=0x7fc588070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:53.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58815d240 0x7fc588070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 >> v1:172.21.15.113:6789/0 conn(0x7fc5880ac860 legacy=0x7fc5880cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc588071240 con 0x7fc5880cf8b0 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fc5800019f0 con 0x7fc5880cf8b0 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 >> v1:172.21.15.72:6789/0 conn(0x7fc5880cf8b0 legacy=0x7fc5880b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc570003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58815d240 0x7fc5700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc570004d00 0x7fc570005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc588070f60 con 0x7fc5700039a0 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57000bb10 con 0x7fc570004d00 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57000bcd0 con 0x7fc58815d240 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.035+0000 7fc586ffd640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc570003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.036+0000 7fc586ffd640 1 -- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58815d240 msgr2=0x7fc5700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.036+0000 7fc586ffd640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58815d240 0x7fc5700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.036+0000 7fc586ffd640 1 -- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc570004d00 msgr2=0x7fc570005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:53.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.036+0000 7fc586ffd640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc570004d00 0x7fc570005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.036+0000 7fc586ffd640 1 -- 172.21.15.72:0/3164376312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570003520 con 0x7fc5700039a0 2023-12-12T09:53:53.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.037+0000 7fc586ffd640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc570003dc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc578009ac0 tx=0x7fc5700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b309b0ba363f4a40 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.037+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57802c710 con 0x7fc5700039a0 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.037+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57802c870 con 0x7fc5700039a0 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.037+0000 7fc585ffb640 1 -- 172.21.15.72:0/3164376312 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57802cc00 con 0x7fc5700039a0 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.038+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3164376312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 msgr2=0x7fc570003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.038+0000 7fc58d5bb640 1 --2- 172.21.15.72:0/3164376312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc570003dc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc578009ac0 tx=0x7fc5700057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.038+0000 7fc5877fe640 1 -- 172.21.15.72:0/3164376312 reap_dead start 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.038+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3164376312 shutdown_connections 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.038+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3164376312 >> 172.21.15.72:0/3164376312 conn(0x7fc58806a880 msgr2=0x7fc58806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.038+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3164376312 shutdown_connections 2023-12-12T09:53:53.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.038+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3164376312 wait complete. 2023-12-12T09:53:53.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.040+0000 7fc58d5bb640 1 Processor -- start 2023-12-12T09:53:53.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.040+0000 7fc58d5bb640 1 -- start start 2023-12-12T09:53:53.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc58d5bb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc5881f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc58d5bb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5880adf10 0x7fc5881476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc58d5bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5880af490 0x7fc5880af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc58d5bb640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880af070 con 0x7fc5700039a0 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc58d5bb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881f2a70 con 0x7fc5880adf10 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc58d5bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880ae3f0 con 0x7fc5880af490 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc586ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc5881f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc586ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc5881f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52276/0 (socket says 172.21.15.72:52276) 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc586ffd640 1 -- 172.21.15.72:0/3094547906 learned_addr learned my addr 172.21.15.72:0/3094547906 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc5877fe640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5880af490 0x7fc5880af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.041+0000 7fc5867fc640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5880adf10 0x7fc5881476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc586ffd640 1 -- 172.21.15.72:0/3094547906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5880af490 msgr2=0x7fc5880af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc586ffd640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5880af490 0x7fc5880af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc586ffd640 1 -- 172.21.15.72:0/3094547906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5880adf10 msgr2=0x7fc5881476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc586ffd640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5880adf10 0x7fc5881476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc586ffd640 1 -- 172.21.15.72:0/3094547906 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5880ae6d0 con 0x7fc5700039a0 2023-12-12T09:53:53.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc586ffd640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc5881f2530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc57802c2b0 tx=0x7fc578004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:53.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc56f7fe640 1 -- 172.21.15.72:0/3094547906 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578003810 con 0x7fc5700039a0 2023-12-12T09:53:53.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc56f7fe640 1 -- 172.21.15.72:0/3094547906 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57802de90 con 0x7fc5700039a0 2023-12-12T09:53:53.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5880ae9b0 con 0x7fc5700039a0 2023-12-12T09:53:53.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc56f7fe640 1 -- 172.21.15.72:0/3094547906 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578003b90 con 0x7fc5700039a0 2023-12-12T09:53:53.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.042+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc588147c00 con 0x7fc5700039a0 2023-12-12T09:53:53.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.043+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5880744c0 con 0x7fc5700039a0 2023-12-12T09:53:53.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.045+0000 7fc56f7fe640 1 -- 172.21.15.72:0/3094547906 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc578003cf0 con 0x7fc5700039a0 2023-12-12T09:53:53.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.045+0000 7fc56f7fe640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc57406ae40 0x7fc57406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.046+0000 7fc56f7fe640 1 -- 172.21.15.72:0/3094547906 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc5780ab670 con 0x7fc5700039a0 2023-12-12T09:53:53.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.046+0000 7fc5867fc640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc57406ae40 0x7fc57406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.047+0000 7fc5867fc640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc57406ae40 0x7fc57406d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc58815c640 tx=0x7fc57c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:53.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.050+0000 7fc56f7fe640 1 -- 172.21.15.72:0/3094547906 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc578076710 con 0x7fc5700039a0 2023-12-12T09:53:53.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.310+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc5880708e0 con 0x7fc5700039a0 2023-12-12T09:53:53.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.315+0000 7fc56f7fe640 1 -- 172.21.15.72:0/3094547906 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc578079d30 con 0x7fc5700039a0 2023-12-12T09:53:53.317 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2023-12-12T09:53:53.318 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQALLXhlQjDrNhAAcu4UiqhTEuide2bwToXy+Q== 2023-12-12T09:53:53.318 INFO:teuthology.orchestra.run.smithi072.stdout: caps mds = "allow" 2023-12-12T09:53:53.318 INFO:teuthology.orchestra.run.smithi072.stdout: caps mon = "allow r" 2023-12-12T09:53:53.318 INFO:teuthology.orchestra.run.smithi072.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-12-12T09:53:53.318 INFO:teuthology.orchestra.run.smithi072.stderr:updated caps for client.0 2023-12-12T09:53:53.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc57406ae40 msgr2=0x7fc57406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc57406ae40 0x7fc57406d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc58815c640 tx=0x7fc57c009040 comp rx=0 tx=0).stop 2023-12-12T09:53:53.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 msgr2=0x7fc5881f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc5881f2530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc57802c2b0 tx=0x7fc578004730 comp rx=0 tx=0).stop 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 shutdown_connections 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc57406ae40 0x7fc57406d300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5880af490 0x7fc5880af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5880adf10 0x7fc5881476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 --2- 172.21.15.72:0/3094547906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5700039a0 0x7fc5881f2530 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 >> 172.21.15.72:0/3094547906 conn(0x7fc58806a880 msgr2=0x7fc58806c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 shutdown_connections 2023-12-12T09:53:53.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.318+0000 7fc58d5bb640 1 -- 172.21.15.72:0/3094547906 wait complete. 2023-12-12T09:53:53.340 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:53:53.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.540+0000 7f0c0b5c9640 1 Processor -- start 2023-12-12T09:53:53.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.541+0000 7f0c0b5c9640 1 -- start start 2023-12-12T09:53:53.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.542+0000 7f0c0b5c9640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0c0415f260 0x7f0c040ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.542+0000 7f0c0b5c9640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c040cde40 con 0x7f0c040708d0 2023-12-12T09:53:53.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.542+0000 7f0c0b5c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c04162b40 con 0x7f0c041626b0 2023-12-12T09:53:53.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.542+0000 7f0c0b5c9640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c040cd040 con 0x7f0c0415f260 2023-12-12T09:53:53.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.542+0000 7f0c09365640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0c0415f260 0x7f0c040ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.543+0000 7f0c09365640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0c0415f260 0x7f0c040ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52282/0 (socket says 172.21.15.72:52282) 2023-12-12T09:53:53.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.543+0000 7f0c09b66640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f0c040708d0 0x7f0c0416b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59972/0 (socket says 172.21.15.72:59972) 2023-12-12T09:53:53.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.543+0000 7f0c09365640 1 -- 172.21.15.72:0/125235741 learned_addr learned my addr 172.21.15.72:0/125235741 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:53.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.543+0000 7f0c09365640 1 -- 172.21.15.72:0/125235741 >> v1:172.21.15.113:6789/0 conn(0x7f0c041626b0 legacy=0x7f0c040701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:53.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.543+0000 7f0c09365640 1 -- 172.21.15.72:0/125235741 >> v1:172.21.15.72:6789/0 conn(0x7f0c040708d0 legacy=0x7f0c0416b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.543+0000 7f0c09365640 1 -- 172.21.15.72:0/125235741 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040cd230 con 0x7f0c0415f260 2023-12-12T09:53:53.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.543+0000 7f0bfbfff640 1 -- 172.21.15.72:0/125235741 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559959514 0 0) 0x7f0c040cde40 con 0x7f0c040708d0 2023-12-12T09:53:53.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.544+0000 7f0c09365640 1 --2- 172.21.15.72:0/125235741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c0415f260 0x7f0c040ccb00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009870 tx=0x7f0c0002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f686b18e2fcf0db4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:53.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.544+0000 7f0bfbfff640 1 -- 172.21.15.72:0/125235741 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00030860 con 0x7f0c0415f260 2023-12-12T09:53:53.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0bfbfff640 1 -- 172.21.15.72:0/125235741 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c000309c0 con 0x7f0c0415f260 2023-12-12T09:53:53.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0bfbfff640 1 -- 172.21.15.72:0/125235741 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00030d50 con 0x7f0c0415f260 2023-12-12T09:53:53.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/125235741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c0415f260 msgr2=0x7f0c040ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0c0b5c9640 1 --2- 172.21.15.72:0/125235741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c0415f260 0x7f0c040ccb00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009870 tx=0x7f0c0002b9a0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/125235741 shutdown_connections 2023-12-12T09:53:53.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0c0b5c9640 1 --2- 172.21.15.72:0/125235741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c0415f260 0x7f0c040ccb00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/125235741 >> 172.21.15.72:0/125235741 conn(0x7f0c040b0a90 msgr2=0x7f0c040b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:53.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/125235741 shutdown_connections 2023-12-12T09:53:53.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.545+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/125235741 wait complete. 2023-12-12T09:53:53.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.547+0000 7f0c0b5c9640 1 Processor -- start 2023-12-12T09:53:53.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.547+0000 7f0c0b5c9640 1 -- start start 2023-12-12T09:53:53.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c0b5c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040708d0 0x7f0c041f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c0b5c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c0415f260 0x7f0c0415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c0b5c9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c041626b0 0x7f0c0415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c0b5c9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c04156660 con 0x7f0c041626b0 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c0b5c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c04156a80 con 0x7f0c040708d0 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c0b5c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c041559e0 con 0x7f0c0415f260 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c09b66640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c041626b0 0x7f0c0415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c09b66640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c041626b0 0x7f0c0415be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52284/0 (socket says 172.21.15.72:52284) 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.548+0000 7f0c09b66640 1 -- 172.21.15.72:0/1397178898 learned_addr learned my addr 172.21.15.72:0/1397178898 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c09365640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040708d0 0x7f0c041f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c09b66640 1 -- 172.21.15.72:0/1397178898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c0415f260 msgr2=0x7f0c0415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c09b66640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c0415f260 0x7f0c0415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c09b66640 1 -- 172.21.15.72:0/1397178898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040708d0 msgr2=0x7f0c041f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c09b66640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040708d0 0x7f0c041f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c09b66640 1 -- 172.21.15.72:0/1397178898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c000094d0 con 0x7f0c041626b0 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c09365640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040708d0 0x7f0c041f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c09b66640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c041626b0 0x7f0c0415be50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00c780 tx=0x7f0bfc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0bf9ffb640 1 -- 172.21.15.72:0/1397178898 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00ce90 con 0x7f0c041626b0 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0bf9ffb640 1 -- 172.21.15.72:0/1397178898 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc00a640 con 0x7f0c041626b0 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0bf9ffb640 1 -- 172.21.15.72:0/1397178898 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00aa70 con 0x7f0c041626b0 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c04155cc0 con 0x7f0c041626b0 2023-12-12T09:53:53.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.549+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c04156280 con 0x7f0c041626b0 2023-12-12T09:53:53.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.550+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c041567a0 con 0x7f0c041626b0 2023-12-12T09:53:53.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.551+0000 7f0bf9ffb640 1 -- 172.21.15.72:0/1397178898 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0bfc00abd0 con 0x7f0c041626b0 2023-12-12T09:53:53.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.552+0000 7f0bf9ffb640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0be806ae40 0x7f0be806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:53.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.552+0000 7f0bf9ffb640 1 -- 172.21.15.72:0/1397178898 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0bfc088d10 con 0x7f0c041626b0 2023-12-12T09:53:53.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.553+0000 7f0c09365640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0be806ae40 0x7f0be806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:53.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.553+0000 7f0c09365640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0be806ae40 0x7f0be806d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0c00036000 tx=0x7f0c0003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:53.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.557+0000 7f0bf9ffb640 1 -- 172.21.15.72:0/1397178898 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bfc08e020 con 0x7f0c041626b0 2023-12-12T09:53:53.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.784+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c040745d0 con 0x7f0c041626b0 2023-12-12T09:53:53.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.785+0000 7f0bf9ffb640 1 -- 172.21.15.72:0/1397178898 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f0bfc018070 con 0x7f0c041626b0 2023-12-12T09:53:53.787 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:53.787 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:53:53.787 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:53:53.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0be806ae40 msgr2=0x7f0be806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0be806ae40 0x7f0be806d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0c00036000 tx=0x7f0c0003b040 comp rx=0 tx=0).stop 2023-12-12T09:53:53.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c041626b0 msgr2=0x7f0c0415be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:53.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c041626b0 0x7f0c0415be50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00c780 tx=0x7f0bfc00cc40 comp rx=0 tx=0).stop 2023-12-12T09:53:53.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 shutdown_connections 2023-12-12T09:53:53.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0be806ae40 0x7f0be806d300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c041626b0 0x7f0c0415be50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c0415f260 0x7f0c0415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 --2- 172.21.15.72:0/1397178898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040708d0 0x7f0c041f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:53.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 >> 172.21.15.72:0/1397178898 conn(0x7f0c040b0a90 msgr2=0x7f0c040b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:53.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 shutdown_connections 2023-12-12T09:53:53.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:53.787+0000 7f0c0b5c9640 1 -- 172.21.15.72:0/1397178898 wait complete. 2023-12-12T09:53:54.211 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-12T09:53:47.254534+0000', 'modified': '2023-12-12T09:53:50.330369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5337}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5337': {'gid': 5337, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.113:6835/211839795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6833', 'nonce': 211839795}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 211839795}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-12T09:53:54.211 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-12T09:53:54.212 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-12T09:53:54.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.432+0000 7fa4b174c640 1 Processor -- start 2023-12-12T09:53:54.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.433+0000 7fa4b174c640 1 -- start start 2023-12-12T09:53:54.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.434+0000 7fa4b174c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac15d580 0x7fa4ac0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.434+0000 7fa4b174c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4ac06c690 0x7fa4ac0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.434+0000 7fa4b174c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ac0b07f0 con 0x7fa4ac0cfc50 2023-12-12T09:53:54.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.434+0000 7fa4b174c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ac15d960 con 0x7fa4ac06c690 2023-12-12T09:53:54.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.434+0000 7fa4b174c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0d05f0 con 0x7fa4ac15d580 2023-12-12T09:53:54.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.434+0000 7fa4aa7fc640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fa4ac0cfc50 0x7fa4ac06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59974/0 (socket says 172.21.15.72:59974) 2023-12-12T09:53:54.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.434+0000 7fa4aa7fc640 1 -- 172.21.15.72:0/1932481208 learned_addr learned my addr 172.21.15.72:0/1932481208 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:54.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac15d580 0x7fa4ac0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/1932481208 >> v2:172.21.15.113:3300/0 conn(0x7fa4ac06c690 0x7fa4ac0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567497474 0 0) 0x7fa4ac0b07f0 con 0x7fa4ac0cfc50 2023-12-12T09:53:54.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa494003520 con 0x7fa4ac0cfc50 2023-12-12T09:53:54.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1657804051 0 0) 0x7fa494003520 con 0x7fa4ac0cfc50 2023-12-12T09:53:54.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac15d580 msgr2=0x7fa4ac0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac15d580 0x7fa4ac0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 >> v2:172.21.15.113:3300/0 conn(0x7fa4ac06c690 msgr2=0x7fa4ac0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 --2- 172.21.15.72:0/1932481208 >> v2:172.21.15.113:3300/0 conn(0x7fa4ac06c690 0x7fa4ac0d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0b0a40 con 0x7fa4ac0cfc50 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fa4a0001670 con 0x7fa4ac0cfc50 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 >> v1:172.21.15.72:6789/0 conn(0x7fa4ac0cfc50 legacy=0x7fa4ac06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac15d580 0x7fa4940039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa494003fd0 0x7fa4940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa494004d00 0x7fa494005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac15d960 con 0x7fa494003fd0 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0d05f0 con 0x7fa494004d00 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.435+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49400bab0 con 0x7fa4ac15d580 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.436+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa494003fd0 0x7fa4940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.436+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac15d580 0x7fa4940039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.436+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa494004d00 0x7fa494005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.436+0000 7fa4aa7fc640 1 -- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac15d580 msgr2=0x7fa4940039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.436+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac15d580 0x7fa4940039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.436+0000 7fa4aa7fc640 1 -- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa494004d00 msgr2=0x7fa494005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.436+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa494004d00 0x7fa494005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.436+0000 7fa4aa7fc640 1 -- 172.21.15.72:0/1932481208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494003520 con 0x7fa494003fd0 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.437+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa494003fd0 0x7fa4940047c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000c240 tx=0x7fa4940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4c91b6bab63970 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.437+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0005ad0 con 0x7fa494003fd0 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.437+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a0005c30 con 0x7fa494003fd0 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.437+0000 7fa4a9ffb640 1 -- 172.21.15.72:0/1932481208 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a002caf0 con 0x7fa494003fd0 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.438+0000 7fa4b174c640 1 -- 172.21.15.72:0/1932481208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa494003fd0 msgr2=0x7fa4940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.438+0000 7fa4b174c640 1 --2- 172.21.15.72:0/1932481208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa494003fd0 0x7fa4940047c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000c240 tx=0x7fa4940057f0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.438+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/1932481208 reap_dead start 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.438+0000 7fa4b174c640 1 -- 172.21.15.72:0/1932481208 shutdown_connections 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.438+0000 7fa4b174c640 1 -- 172.21.15.72:0/1932481208 >> 172.21.15.72:0/1932481208 conn(0x7fa4ac0d0e90 msgr2=0x7fa4ac0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:54.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.438+0000 7fa4b174c640 1 -- 172.21.15.72:0/1932481208 shutdown_connections 2023-12-12T09:53:54.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.438+0000 7fa4b174c640 1 -- 172.21.15.72:0/1932481208 wait complete. 2023-12-12T09:53:54.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.440+0000 7fa4b174c640 1 Processor -- start 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.440+0000 7fa4b174c640 1 -- start start 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4b174c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa494003fd0 0x7fa4ac157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4b174c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac0c53d0 0x7fa4ac0b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4b174c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ac0c5850 0x7fa4ac0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4b174c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac157b70 con 0x7fa4ac0c5850 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4b174c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac158c60 con 0x7fa494003fd0 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4b174c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0c5e20 con 0x7fa4ac0c53d0 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4ab7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ac0c5850 0x7fa4ac0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4ab7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ac0c5850 0x7fa4ac0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52300/0 (socket says 172.21.15.72:52300) 2023-12-12T09:53:54.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/3162399872 learned_addr learned my addr 172.21.15.72:0/3162399872 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:54.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac0c53d0 0x7fa4ac0b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/3162399872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac0c53d0 msgr2=0x7fa4ac0b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac0c53d0 0x7fa4ac0b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/3162399872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa494003fd0 msgr2=0x7fa4ac157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa494003fd0 0x7fa4ac157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.441+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/3162399872 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0c6100 con 0x7fa4ac0c5850 2023-12-12T09:53:54.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.442+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ac0c5850 0x7fa4ac0b5c60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0717f0 tx=0x7fa4a4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.442+0000 7fa4937fe640 1 -- 172.21.15.72:0/3162399872 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4009e70 con 0x7fa4ac0c5850 2023-12-12T09:53:54.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.442+0000 7fa4937fe640 1 -- 172.21.15.72:0/3162399872 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a400f050 con 0x7fa4ac0c5850 2023-12-12T09:53:54.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.442+0000 7fa4b174c640 1 -- 172.21.15.72:0/3162399872 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac0c63e0 con 0x7fa4ac0c5850 2023-12-12T09:53:54.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.442+0000 7fa4937fe640 1 -- 172.21.15.72:0/3162399872 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4014650 con 0x7fa4ac0c5850 2023-12-12T09:53:54.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.442+0000 7fa4b174c640 1 -- 172.21.15.72:0/3162399872 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac0b6420 con 0x7fa4ac0c5850 2023-12-12T09:53:54.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.444+0000 7fa4937fe640 1 -- 172.21.15.72:0/3162399872 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa4a40147b0 con 0x7fa4ac0c5850 2023-12-12T09:53:54.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.445+0000 7fa4937fe640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa49806ae40 0x7fa49806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.445+0000 7fa4937fe640 1 -- 172.21.15.72:0/3162399872 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa4a4089d60 con 0x7fa4ac0c5850 2023-12-12T09:53:54.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.445+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa49806ae40 0x7fa49806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.446+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa49806ae40 0x7fa49806d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa49c006a90 tx=0x7fa49c006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.446+0000 7fa4b43be740 1 Processor -- start 2023-12-12T09:53:54.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.446+0000 7fa4b43be740 1 -- start start 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b21b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eb2c0 0x561c9b218280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b1eeb90 0x561c9b2187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4b43be740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9b15df40 con 0x561c9b1eeb90 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4b43be740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9b15b630 con 0x561c9b1eb2c0 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4b43be740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c9b156f20 con 0x561c9b21b010 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4ab7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b1eeb90 0x561c9b2187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4ab7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b1eeb90 0x561c9b2187c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52302/0 (socket says 172.21.15.72:52302) 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/3827849966 learned_addr learned my addr 172.21.15.72:0/3827849966 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.447+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eb2c0 0x561c9b218280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b21b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 msgr2=0x561c9b21b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b21b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eb2c0 msgr2=0x561c9b218280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eb2c0 0x561c9b218280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/3827849966 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c9b119910 con 0x561c9b1eeb90 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b21b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eb2c0 0x561c9b218280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b1eeb90 0x561c9b2187c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4088020 tx=0x7fa4a4058250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3739a079f3c78d36 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa490ff9640 1 -- 172.21.15.72:0/3827849966 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a405ca40 con 0x561c9b1eeb90 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa490ff9640 1 -- 172.21.15.72:0/3827849966 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a4060bf0 con 0x561c9b1eeb90 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.448+0000 7fa490ff9640 1 -- 172.21.15.72:0/3827849966 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40701c0 con 0x561c9b1eeb90 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 -- 172.21.15.72:0/3827849966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b1eeb90 msgr2=0x561c9b2187c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b1eeb90 0x561c9b2187c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4088020 tx=0x7fa4a4058250 comp rx=0 tx=0).stop 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 -- 172.21.15.72:0/3827849966 shutdown_connections 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b1eeb90 0x561c9b2187c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eb2c0 0x561c9b218280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3827849966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b21b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 -- 172.21.15.72:0/3827849966 >> 172.21.15.72:0/3827849966 conn(0x561c9b1ed530 msgr2=0x561c9b235e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:54.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 -- 172.21.15.72:0/3827849966 shutdown_connections 2023-12-12T09:53:54.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.449+0000 7fa4b43be740 1 -- 172.21.15.72:0/3827849966 wait complete. 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.450+0000 7fa4b43be740 1 -- start start 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.450+0000 7fa4b43be740 1 Processor -- start 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b21e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b206190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4b43be740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9b0f32f0 con 0x561c9b21a010 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4b43be740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9b110c70 con 0x561c9b1eeb90 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4b43be740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c9aeb9f20 con 0x561c9b21b010 2023-12-12T09:53:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4b43be740 10 client.0 _ll_register_callbacks cb 0x561c9b1153b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4ab7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4ab7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52308/0 (socket says 172.21.15.72:52308) 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/1158776059 learned_addr learned my addr 172.21.15.72:0/1158776059 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 msgr2=0x561c9b206190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b206190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 msgr2=0x561c9b21e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b21e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.451+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/1158776059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac0b6420 con 0x561c9b21a010 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2066d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009fd0 tx=0x7fa4a4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57561642cc48188d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa489ffb640 1 -- 172.21.15.72:0/1158776059 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4057fe0 con 0x561c9b21a010 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa489ffb640 1 -- 172.21.15.72:0/1158776059 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa4a4060760 con 0x561c9b21a010 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 1 -- 172.21.15.72:0/1158776059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c9b14fdb0 con 0x561c9b21a010 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa489ffb640 1 -- 172.21.15.72:0/1158776059 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4a4060940 con 0x561c9b21a010 2023-12-12T09:53:54.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 10 client.5520 fetch_fsmap learned FSMap version 16 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 1 -- 172.21.15.72:0/1158776059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x561c9b1555a0 con 0x561c9b21a010 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa489ffb640 1 -- 172.21.15.72:0/1158776059 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561c9b14fdb0 con 0x561c9b21a010 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa489ffb640 1 -- 172.21.15.72:0/1158776059 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa4a40546e0 con 0x561c9b21a010 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 10 client.5520 fetch_fsmap finished waiting for FSMap version 16 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 10 client.5520 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 20 client.5520 populate_metadata read hostname 'smithi072' 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] conn(0x561c9b24a1f0 0x561c9b204c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 4 client.5520 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.452+0000 7fa4b43be740 1 -- 172.21.15.72:0/1158776059 --> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c9b14fad0 con 0x561c9b24a1f0 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.453+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] conn(0x561c9b24a1f0 0x561c9b204c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.454+0000 7fa489ffb640 10 client.5520 ms_handle_connect on v2:172.21.15.113:6833/211839795 2023-12-12T09:53:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.454+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] conn(0x561c9b24a1f0 0x561c9b204c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e24d5ba70058e2a server_cookie=edb5b5b0b284a32 in_seq=0 out_seq=0 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa489ffb640 1 -- 172.21.15.72:0/1158776059 <== mds.0 v2:172.21.15.113:6833/211839795 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561c9b14fad0 con 0x561c9b24a1f0 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa489ffb640 10 client.5520 handle_command_reply: tid=0 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 client.5520 shutdown 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 -- 172.21.15.72:0/1158776059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 msgr2=0x561c9b2066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2066d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009fd0 tx=0x7fa4a4007f90 comp rx=0 tx=0).stop 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 -- 172.21.15.72:0/1158776059 shutdown_connections 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] conn(0x561c9b24a1f0 0x561c9b204c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2066d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b206190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 --2- 172.21.15.72:0/1158776059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b21e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 -- 172.21.15.72:0/1158776059 >> 172.21.15.72:0/1158776059 conn(0x561c9b1eb2c0 msgr2=0x561c9b21ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa489ffb640 0 client.5520 ms_handle_reset on v2:172.21.15.113:6833/211839795 2023-12-12T09:53:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 -- 172.21.15.72:0/1158776059 shutdown_connections 2023-12-12T09:53:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.456+0000 7fa4b43be740 1 -- 172.21.15.72:0/1158776059 wait complete. 2023-12-12T09:53:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.457+0000 7fa4b43be740 20 client.5520 trim_cache size 0 max 16384 2023-12-12T09:53:54.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.473+0000 7fa4b43be740 1 Processor -- start 2023-12-12T09:53:54.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.473+0000 7fa4b43be740 1 -- start start 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b239330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21a010 0x561c9b238660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21b010 0x561c9b238ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4b43be740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9b1ee670 con 0x561c9b21b010 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4b43be740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9b237b30 con 0x561c9b1eeb90 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4b43be740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c9b22d780 con 0x561c9b21a010 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4ab7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21b010 0x561c9b238ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4ab7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21b010 0x561c9b238ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52320/0 (socket says 172.21.15.72:52320) 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.474+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/2577031009 learned_addr learned my addr 172.21.15.72:0/2577031009 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/2577031009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21a010 msgr2=0x561c9b238660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/2577031009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21a010 0x561c9b238660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/2577031009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 msgr2=0x561c9b239330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/2577031009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b239330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4ab7fe640 1 -- 172.21.15.72:0/2577031009 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c9b209050 con 0x561c9b21b010 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4ab7fe640 1 --2- 172.21.15.72:0/2577031009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21b010 0x561c9b238ba0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4056c40 tx=0x7fa4a4083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef5aa0fc90d1fe0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4917fa640 1 -- 172.21.15.72:0/2577031009 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a405ca40 con 0x561c9b21b010 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4917fa640 1 -- 172.21.15.72:0/2577031009 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a4060760 con 0x561c9b21b010 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.475+0000 7fa4917fa640 1 -- 172.21.15.72:0/2577031009 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4057de0 con 0x561c9b21b010 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 -- 172.21.15.72:0/2577031009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21b010 msgr2=0x561c9b238ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 --2- 172.21.15.72:0/2577031009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21b010 0x561c9b238ba0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4056c40 tx=0x7fa4a4083040 comp rx=0 tx=0).stop 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 -- 172.21.15.72:0/2577031009 shutdown_connections 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 --2- 172.21.15.72:0/2577031009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21b010 0x561c9b238ba0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 --2- 172.21.15.72:0/2577031009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21a010 0x561c9b238660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 --2- 172.21.15.72:0/2577031009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b239330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 -- 172.21.15.72:0/2577031009 >> 172.21.15.72:0/2577031009 conn(0x561c9b1eb2c0 msgr2=0x561c9b21e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:54.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 -- 172.21.15.72:0/2577031009 shutdown_connections 2023-12-12T09:53:54.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.476+0000 7fa4b43be740 1 -- 172.21.15.72:0/2577031009 wait complete. 2023-12-12T09:53:54.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.477+0000 7fa4b43be740 1 -- start start 2023-12-12T09:53:54.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.477+0000 7fa4b43be740 1 Processor -- start 2023-12-12T09:53:54.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b2483b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4b43be740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b2a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4b43be740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9b2a19a0 con 0x561c9b21a010 2023-12-12T09:53:54.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4b43be740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9b237800 con 0x561c9b1eeb90 2023-12-12T09:53:54.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4b43be740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c9b22d580 con 0x561c9b21b010 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4b43be740 10 client.0 _ll_register_callbacks cb 0x7fa4a4058790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4aa7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4aa7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52328/0 (socket says 172.21.15.72:52328) 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4aa7fc640 1 -- 172.21.15.72:0/3281262115 learned_addr learned my addr 172.21.15.72:0/3281262115 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.478+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b2483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4aa7fc640 1 -- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 msgr2=0x561c9b2a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b2a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4aa7fc640 1 -- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 msgr2=0x561c9b2483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b2483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4aa7fc640 1 -- 172.21.15.72:0/3281262115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c9b1555a0 con 0x561c9b21a010 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4aa7fc640 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2488f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa4a002c450 tx=0x7fa4a0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d338a7fd0c5a6f48 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa489ffb640 1 -- 172.21.15.72:0/3281262115 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a002ccc0 con 0x561c9b21a010 2023-12-12T09:53:54.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa489ffb640 1 -- 172.21.15.72:0/3281262115 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa4a003c070 con 0x561c9b21a010 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4b43be740 1 -- 172.21.15.72:0/3281262115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4a4060bf0 con 0x561c9b21a010 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa489ffb640 1 -- 172.21.15.72:0/3281262115 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4a002ce20 con 0x561c9b21a010 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4b43be740 10 client.5526 fetch_fsmap learned FSMap version 16 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.479+0000 7fa4b43be740 1 -- 172.21.15.72:0/3281262115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x561c9b0f2b80 con 0x561c9b21a010 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.480+0000 7fa489ffb640 1 -- 172.21.15.72:0/3281262115 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a4060bf0 con 0x561c9b21a010 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.480+0000 7fa489ffb640 1 -- 172.21.15.72:0/3281262115 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa4a00047e0 con 0x561c9b21a010 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.480+0000 7fa4b43be740 10 client.5526 fetch_fsmap finished waiting for FSMap version 16 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.480+0000 7fa4b43be740 10 client.5526 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.480+0000 7fa4b43be740 20 client.5526 populate_metadata read hostname 'smithi072' 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.480+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] conn(0x561c9b2a1110 0x561c9b217ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.480+0000 7fa4b43be740 4 client.5526 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.480+0000 7fa4b43be740 1 -- 172.21.15.72:0/3281262115 --> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561c9ae576c0 con 0x561c9b2a1110 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.481+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] conn(0x561c9b2a1110 0x561c9b217ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.481+0000 7fa4aaffd640 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] conn(0x561c9b2a1110 0x561c9b217ee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f19f153e3fdee98c server_cookie=22f440bf97bd5d02 in_seq=0 out_seq=0 2023-12-12T09:53:54.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.481+0000 7fa489ffb640 10 client.5526 ms_handle_connect on v2:172.21.15.113:6833/211839795 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa489ffb640 1 -- 172.21.15.72:0/3281262115 <== mds.0 v2:172.21.15.113:6833/211839795 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x561c9ae576c0 con 0x561c9b2a1110 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa489ffb640 10 client.5526 handle_command_reply: tid=0 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 client.5526 shutdown 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 -- 172.21.15.72:0/3281262115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 msgr2=0x561c9b2488f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2488f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa4a002c450 tx=0x7fa4a0001e50 comp rx=0 tx=0).stop 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 -- 172.21.15.72:0/3281262115 shutdown_connections 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:6833/211839795,v1:172.21.15.113:6835/211839795] conn(0x561c9b2a1110 0x561c9b217ee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561c9b21b010 0x561c9b2a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561c9b21a010 0x561c9b2488f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa489ffb640 0 client.5526 ms_handle_reset on v2:172.21.15.113:6833/211839795 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 --2- 172.21.15.72:0/3281262115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c9b1eeb90 0x561c9b2483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 -- 172.21.15.72:0/3281262115 >> 172.21.15.72:0/3281262115 conn(0x561c9b1eb2c0 msgr2=0x561c9b238620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 -- 172.21.15.72:0/3281262115 shutdown_connections 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.482+0000 7fa4b43be740 1 -- 172.21.15.72:0/3281262115 wait complete. 2023-12-12T09:53:54.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b43be740 20 client.5526 trim_cache size 0 max 16384 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "bd07f350-42f2-4d45-9a3d-13364a8566fe", 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 5337, 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 6.6604179950000004, 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 16, 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 38, 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 38, 2023-12-12T09:53:54.488 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 15.512559037000001 2023-12-12T09:53:54.489 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:53:54.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 -- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa49806ae40 msgr2=0x7fa49806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa49806ae40 0x7fa49806d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa49c006a90 tx=0x7fa49c006a20 comp rx=0 tx=0).stop 2023-12-12T09:53:54.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 -- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ac0c5850 msgr2=0x7fa4ac0b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ac0c5850 0x7fa4ac0b5c60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0717f0 tx=0x7fa4a4007700 comp rx=0 tx=0).stop 2023-12-12T09:53:54.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 -- 172.21.15.72:0/3162399872 shutdown_connections 2023-12-12T09:53:54.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa49806ae40 0x7fa49806d300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ac0c5850 0x7fa4ac0b5c60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4ac0c53d0 0x7fa4ac0b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 --2- 172.21.15.72:0/3162399872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa494003fd0 0x7fa4ac157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 -- 172.21.15.72:0/3162399872 >> 172.21.15.72:0/3162399872 conn(0x7fa4ac0d0e90 msgr2=0x7fa4ac06bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:54.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 -- 172.21.15.72:0/3162399872 shutdown_connections 2023-12-12T09:53:54.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.483+0000 7fa4b174c640 1 -- 172.21.15.72:0/3162399872 wait complete. 2023-12-12T09:53:54.501 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-12T09:53:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.701+0000 7fc088a00640 1 Processor -- start 2023-12-12T09:53:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.701+0000 7fc088a00640 1 -- start start 2023-12-12T09:53:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.703+0000 7fc088a00640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc0840cd000 0x7fc08406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.703+0000 7fc088a00640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08406d510 con 0x7fc084146ba0 2023-12-12T09:53:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.703+0000 7fc088a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08406d7f0 con 0x7fc0841467c0 2023-12-12T09:53:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.703+0000 7fc088a00640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0840ac860 con 0x7fc0840cd000 2023-12-12T09:53:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.703+0000 7fc081d9b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc0840cd000 0x7fc08406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.703+0000 7fc082d9d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc084146ba0 0x7fc08406cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:59978/0 (socket says 172.21.15.72:59978) 2023-12-12T09:53:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.703+0000 7fc082d9d640 1 -- 172.21.15.72:0/1900002873 learned_addr learned my addr 172.21.15.72:0/1900002873 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:53:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.704+0000 7fc081d9b640 1 -- 172.21.15.72:0/1900002873 >> v1:172.21.15.113:6789/0 conn(0x7fc0841467c0 legacy=0x7fc0840cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.704+0000 7fc081d9b640 1 -- 172.21.15.72:0/1900002873 >> v1:172.21.15.72:6789/0 conn(0x7fc084146ba0 legacy=0x7fc08406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.704+0000 7fc081d9b640 1 -- 172.21.15.72:0/1900002873 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08406dad0 con 0x7fc0840cd000 2023-12-12T09:53:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.704+0000 7fc08159a640 1 -- 172.21.15.72:0/1900002873 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93045970 0 0) 0x7fc08406d510 con 0x7fc084146ba0 2023-12-12T09:53:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.705+0000 7fc081d9b640 1 --2- 172.21.15.72:0/1900002873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0840cd000 0x7fc08406c7e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc0700097a0 tx=0x7fc07002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d566c9ab49708b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.705+0000 7fc08159a640 1 -- 172.21.15.72:0/1900002873 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070031830 con 0x7fc0840cd000 2023-12-12T09:53:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.705+0000 7fc08159a640 1 -- 172.21.15.72:0/1900002873 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc070031990 con 0x7fc0840cd000 2023-12-12T09:53:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.705+0000 7fc08159a640 1 -- 172.21.15.72:0/1900002873 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070031d20 con 0x7fc0840cd000 2023-12-12T09:53:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.705+0000 7fc088a00640 1 -- 172.21.15.72:0/1900002873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0840cd000 msgr2=0x7fc08406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.705+0000 7fc088a00640 1 --2- 172.21.15.72:0/1900002873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0840cd000 0x7fc08406c7e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc0700097a0 tx=0x7fc07002b8d0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.706+0000 7fc088a00640 1 -- 172.21.15.72:0/1900002873 shutdown_connections 2023-12-12T09:53:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.706+0000 7fc088a00640 1 --2- 172.21.15.72:0/1900002873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0840cd000 0x7fc08406c7e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.706+0000 7fc088a00640 1 -- 172.21.15.72:0/1900002873 >> 172.21.15.72:0/1900002873 conn(0x7fc0840b0a90 msgr2=0x7fc0840b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:54.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.706+0000 7fc088a00640 1 -- 172.21.15.72:0/1900002873 shutdown_connections 2023-12-12T09:53:54.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.706+0000 7fc088a00640 1 -- 172.21.15.72:0/1900002873 wait complete. 2023-12-12T09:53:54.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.707+0000 7fc088a00640 1 Processor -- start 2023-12-12T09:53:54.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.708+0000 7fc088a00640 1 -- start start 2023-12-12T09:53:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc088a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0840cd000 0x7fc084157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc088a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0841467c0 0x7fc084156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc088a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc084146ba0 0x7fc0840b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc088a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc084156fc0 con 0x7fc0841467c0 2023-12-12T09:53:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc088a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840b3520 con 0x7fc0840cd000 2023-12-12T09:53:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc088a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840b3800 con 0x7fc084146ba0 2023-12-12T09:53:54.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc081d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0841467c0 0x7fc084156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc081d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0841467c0 0x7fc084156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52346/0 (socket says 172.21.15.72:52346) 2023-12-12T09:53:54.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc081d9b640 1 -- 172.21.15.72:0/1412743880 learned_addr learned my addr 172.21.15.72:0/1412743880 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:54.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc081d9b640 1 -- 172.21.15.72:0/1412743880 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc084146ba0 msgr2=0x7fc0840b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc081d9b640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc084146ba0 0x7fc0840b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc081d9b640 1 -- 172.21.15.72:0/1412743880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0840cd000 msgr2=0x7fc084157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc081d9b640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0840cd000 0x7fc084157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.709+0000 7fc081d9b640 1 -- 172.21.15.72:0/1412743880 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc070009400 con 0x7fc0841467c0 2023-12-12T09:53:54.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.710+0000 7fc081d9b640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0841467c0 0x7fc084156a80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc0700097a0 tx=0x7fc0700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.710+0000 7fc07affd640 1 -- 172.21.15.72:0/1412743880 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07003ca80 con 0x7fc0841467c0 2023-12-12T09:53:54.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.710+0000 7fc07affd640 1 -- 172.21.15.72:0/1412743880 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07003cbe0 con 0x7fc0841467c0 2023-12-12T09:53:54.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.710+0000 7fc07affd640 1 -- 172.21.15.72:0/1412743880 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070004030 con 0x7fc0841467c0 2023-12-12T09:53:54.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.710+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0840b3ae0 con 0x7fc0841467c0 2023-12-12T09:53:54.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.710+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08415bff0 con 0x7fc0841467c0 2023-12-12T09:53:54.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.711+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc084067910 con 0x7fc0841467c0 2023-12-12T09:53:54.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.712+0000 7fc07affd640 1 -- 172.21.15.72:0/1412743880 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc070004190 con 0x7fc0841467c0 2023-12-12T09:53:54.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.713+0000 7fc07affd640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc06c06ae40 0x7fc06c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:54.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.713+0000 7fc07affd640 1 -- 172.21.15.72:0/1412743880 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc0700abf30 con 0x7fc0841467c0 2023-12-12T09:53:54.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.714+0000 7fc08259c640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc06c06ae40 0x7fc06c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:54.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.714+0000 7fc08259c640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc06c06ae40 0x7fc06c06d300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc074003cc0 tx=0x7fc0740079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:54.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.718+0000 7fc07affd640 1 -- 172.21.15.72:0/1412743880 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc070076e90 con 0x7fc0841467c0 2023-12-12T09:53:54.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.950+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc084067910 con 0x7fc0841467c0 2023-12-12T09:53:54.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.951+0000 7fc07affd640 1 -- 172.21.15.72:0/1412743880 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc07007a4b0 con 0x7fc0841467c0 2023-12-12T09:53:54.953 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-12T09:53:54.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.953+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc06c06ae40 msgr2=0x7fc06c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc06c06ae40 0x7fc06c06d300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc074003cc0 tx=0x7fc0740079b0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0841467c0 msgr2=0x7fc084156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:54.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0841467c0 0x7fc084156a80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc0700097a0 tx=0x7fc0700046c0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 shutdown_connections 2023-12-12T09:53:54.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc06c06ae40 0x7fc06c06d300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc084146ba0 0x7fc0840b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0841467c0 0x7fc084156a80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 --2- 172.21.15.72:0/1412743880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0840cd000 0x7fc084157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:54.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 >> 172.21.15.72:0/1412743880 conn(0x7fc0840b0a90 msgr2=0x7fc084145730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:54.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 shutdown_connections 2023-12-12T09:53:54.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:54.954+0000 7fc088a00640 1 -- 172.21.15.72:0/1412743880 wait complete. 2023-12-12T09:53:54.975 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-12T09:53:54.975 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:53:54.975 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2023-12-12T09:53:54.975 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-12T09:53:54.975 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-12T09:53:54.976 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-12T09:53:54.976 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-12T09:53:54.976 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:53:54.976 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:53:54.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:54.976 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2023-12-12T09:53:55.029 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 42267sec preferred_lft 42267sec 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 3e:b1:ce:9c:47:2c brd ff:ff:ff:ff:ff:ff 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:55.030 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::c4c0:7aff:fe81:b145/64 scope link 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 3e:b1:ce:9c:47:2c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::3cb1:ceff:fe9c:472c/64 scope link 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 86:f9:73:7a:97:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::84f9:73ff:fe7a:975d/64 scope link 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:53:55.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:55.031 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:53:55.120 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-12-12T09:53:55.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:55.124 DEBUG:teuthology.orchestra.run.smithi072:> route 2023-12-12T09:53:55.183 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2023-12-12T09:53:55.183 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:53:55.183 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-12T09:53:55.183 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-12T09:53:55.183 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:53:55.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:55.184 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:53:55.184 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:53:55.184 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-12T09:53:55.184 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-12T09:53:55.184 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-12T09:53:55.184 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:53:55.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:55.364 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:53:55.416 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-12T09:53:55.416 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-12T09:53:55.417 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:53:55.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:55.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2023-12-12T09:53:55.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:53:55.517 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-12T09:53:55.565 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:53:55.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:53:55.566 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-12T09:53:55.613 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-12T09:53:55.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:55.613 DEBUG:teuthology.orchestra.run.smithi072:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:55.800 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:55.800 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:53:55.837 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:53:55.837 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:53:55.838 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.72:3300,172.21.15.113:3300,172.21.15.113:3301" 2023-12-12T09:53:55.838 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with new device syntax: 0@bd07f350-42f2-4d45-9a3d-13364a8566fe.cephfs=/ 2023-12-12T09:53:55.838 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.72:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2023-12-12T09:53:55.840 INFO:teuthology.orchestra.run.smithi072.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-12T09:53:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: You just mounted a file system that supports labels which does not 2023-12-12T09:53:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-12T09:53:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: applications will generate AVC messages and not be allowed access to 2023-12-12T09:53:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-12T09:53:55.845 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-12T09:53:55.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:55.846 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-12T09:53:55.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:55.927 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:53:56.006 INFO:teuthology.orchestra.run.smithi072.stdout:client5538 2023-12-12T09:53:56.006 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:53:56.007 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:53:56.063 DEBUG:teuthology.orchestra.run.smithi072:> stdin-killer -- python3 -c ' 2023-12-12T09:53:56.063 DEBUG:teuthology.orchestra.run.smithi072:> import os 2023-12-12T09:53:56.063 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:53:56.063 DEBUG:teuthology.orchestra.run.smithi072:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-12T09:53:56.063 DEBUG:teuthology.orchestra.run.smithi072:> subdir = "delete_me" 2023-12-12T09:53:56.063 DEBUG:teuthology.orchestra.run.smithi072:> size = 1024 2023-12-12T09:53:56.064 DEBUG:teuthology.orchestra.run.smithi072:> file_count = 1000 2023-12-12T09:53:56.064 DEBUG:teuthology.orchestra.run.smithi072:> os.mkdir(os.path.join(mountpoint, subdir)) 2023-12-12T09:53:56.064 DEBUG:teuthology.orchestra.run.smithi072:> for i in range(0, file_count): 2023-12-12T09:53:56.064 DEBUG:teuthology.orchestra.run.smithi072:> filename = "{0}_{1}.bin".format(i, size) 2023-12-12T09:53:56.064 DEBUG:teuthology.orchestra.run.smithi072:> with open(os.path.join(mountpoint, subdir, filename), '"'"'w'"'"') as f: 2023-12-12T09:53:56.064 DEBUG:teuthology.orchestra.run.smithi072:> f.write(size * '"'"'x'"'"') 2023-12-12T09:53:56.064 DEBUG:teuthology.orchestra.run.smithi072:> ' 2023-12-12T09:53:57.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-12T09:53:57.266 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:53:57.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.523+0000 7fadcb22b640 1 Processor -- start 2023-12-12T09:53:57.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.523+0000 7fadcb22b640 1 -- start start 2023-12-12T09:53:57.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.525+0000 7fadcb22b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fadc40c3ea0 0x7fadc40c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:57.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.525+0000 7fadcb22b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:57.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.525+0000 7fadcb22b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc40b1160 con 0x7fadc406bab0 2023-12-12T09:53:57.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.525+0000 7fadcb22b640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc40afac0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.525+0000 7fadcb22b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40c47c0 con 0x7fadc40b15f0 2023-12-12T09:53:57.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.525+0000 7fadc8fc7640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fadc40c3ea0 0x7fadc40c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.525+0000 7fadc8fc7640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fadc40c3ea0 0x7fadc40c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57458/0 (socket says 172.21.15.72:57458) 2023-12-12T09:53:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.525+0000 7fadc8fc7640 1 -- 172.21.15.72:0/1767486487 learned_addr learned my addr 172.21.15.72:0/1767486487 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.526+0000 7fadc8fc7640 1 -- 172.21.15.72:0/1767486487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 msgr2=0x7fadc406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.526+0000 7fadc3fff640 1 --2- 172.21.15.72:0/1767486487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.526+0000 7fadc8fc7640 1 --2- 172.21.15.72:0/1767486487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc406b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.526+0000 7fadc8fc7640 1 -- 172.21.15.72:0/1767486487 >> v1:172.21.15.113:6789/0 conn(0x7fadc406bab0 legacy=0x7fadc4165320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:57.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.526+0000 7fadc8fc7640 1 -- 172.21.15.72:0/1767486487 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc40b0270 con 0x7fadc40c3ea0 2023-12-12T09:53:57.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.526+0000 7fadc3fff640 1 --2- 172.21.15.72:0/1767486487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:53:57.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.527+0000 7fadc8fc7640 1 --2- 172.21.15.72:0/1767486487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40c4280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009870 tx=0x7fadb802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624dd8f3c086fdf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:57.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.528+0000 7fadc37fe640 1 -- 172.21.15.72:0/1767486487 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8031640 con 0x7fadc40c3ea0 2023-12-12T09:53:57.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.528+0000 7fadc37fe640 1 -- 172.21.15.72:0/1767486487 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb80317a0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.528+0000 7fadc37fe640 1 -- 172.21.15.72:0/1767486487 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8031b30 con 0x7fadc40c3ea0 2023-12-12T09:53:57.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.528+0000 7fadcb22b640 1 -- 172.21.15.72:0/1767486487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 msgr2=0x7fadc40c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:57.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.528+0000 7fadcb22b640 1 --2- 172.21.15.72:0/1767486487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40c4280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009870 tx=0x7fadb802b8f0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.529+0000 7fadcb22b640 1 -- 172.21.15.72:0/1767486487 shutdown_connections 2023-12-12T09:53:57.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.529+0000 7fadcb22b640 1 --2- 172.21.15.72:0/1767486487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.529+0000 7fadcb22b640 1 --2- 172.21.15.72:0/1767486487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40c4280 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.529+0000 7fadcb22b640 1 -- 172.21.15.72:0/1767486487 >> 172.21.15.72:0/1767486487 conn(0x7fadc40d0ed0 msgr2=0x7fadc40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:57.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.529+0000 7fadcb22b640 1 -- 172.21.15.72:0/1767486487 shutdown_connections 2023-12-12T09:53:57.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.529+0000 7fadcb22b640 1 -- 172.21.15.72:0/1767486487 wait complete. 2023-12-12T09:53:57.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.530+0000 7fadcb22b640 1 Processor -- start 2023-12-12T09:53:57.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.531+0000 7fadcb22b640 1 -- start start 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.531+0000 7fadcb22b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc406bab0 0x7fadc414e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.531+0000 7fadcb22b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc40b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadcb22b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadcb22b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc414ec10 con 0x7fadc40c3ea0 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadcb22b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40c54e0 con 0x7fadc406bab0 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadcb22b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40c57c0 con 0x7fadc40b15f0 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc97c8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc97c8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57462/0 (socket says 172.21.15.72:57462) 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc97c8640 1 -- 172.21.15.72:0/468551819 learned_addr learned my addr 172.21.15.72:0/468551819 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc8fc7640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc406bab0 0x7fadc414e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:57.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc3fff640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc40b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:57.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc97c8640 1 -- 172.21.15.72:0/468551819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 msgr2=0x7fadc40b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc97c8640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc40b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc97c8640 1 -- 172.21.15.72:0/468551819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc406bab0 msgr2=0x7fadc414e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc97c8640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc406bab0 0x7fadc414e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.532+0000 7fadc97c8640 1 -- 172.21.15.72:0/468551819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80094d0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.533+0000 7fadc97c8640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40b42b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00b970 tx=0x7fadbc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.533+0000 7fadc17fa640 1 -- 172.21.15.72:0/468551819 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00cb40 con 0x7fadc40c3ea0 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.533+0000 7fadc17fa640 1 -- 172.21.15.72:0/468551819 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc00cca0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.533+0000 7fadc17fa640 1 -- 172.21.15.72:0/468551819 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc014680 con 0x7fadc40c3ea0 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.533+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc40c5aa0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.533+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc40c6060 con 0x7fadc40c3ea0 2023-12-12T09:53:57.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.534+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc40c6280 con 0x7fadc40c3ea0 2023-12-12T09:53:57.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.535+0000 7fadc17fa640 1 -- 172.21.15.72:0/468551819 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadbc01d3b0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.536+0000 7fadc17fa640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fada806ae40 0x7fada806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:57.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.536+0000 7fadc17fa640 1 -- 172.21.15.72:0/468551819 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fadbc089560 con 0x7fadc40c3ea0 2023-12-12T09:53:57.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.536+0000 7fadc8fc7640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fada806ae40 0x7fada806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:57.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.537+0000 7fadc8fc7640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fada806ae40 0x7fada806d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fadb8036000 tx=0x7fadb803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:57.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.541+0000 7fadc17fa640 1 -- 172.21.15.72:0/468551819 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadbc0544b0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.768+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadc40745d0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.769+0000 7fadc17fa640 1 -- 172.21.15.72:0/468551819 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fadbc057ad0 con 0x7fadc40c3ea0 2023-12-12T09:53:57.771 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:53:57.771 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:53:57.772 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.771+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fada806ae40 msgr2=0x7fada806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.771+0000 7fadcb22b640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fada806ae40 0x7fada806d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fadb8036000 tx=0x7fadb803c040 comp rx=0 tx=0).stop 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 msgr2=0x7fadc40b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40b42b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00b970 tx=0x7fadbc00be30 comp rx=0 tx=0).stop 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 shutdown_connections 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fada806ae40 0x7fada806d300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadc40c3ea0 0x7fadc40b42b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadc40b15f0 0x7fadc40b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 --2- 172.21.15.72:0/468551819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadc406bab0 0x7fadc414e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:57.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 >> 172.21.15.72:0/468551819 conn(0x7fadc40d0ed0 msgr2=0x7fadc40b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:57.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 shutdown_connections 2023-12-12T09:53:57.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:57.772+0000 7fadcb22b640 1 -- 172.21.15.72:0/468551819 wait complete. 2023-12-12T09:53:57.793 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-12-12T09:53:58.373 INFO:teuthology.orchestra.run.smithi113.stdout:{"message":"","return_code":0} 2023-12-12T09:53:58.373 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-12-12T09:53:58.374 DEBUG:teuthology.orchestra.run.smithi072:> stdin-killer -- python3 -c ' 2023-12-12T09:53:58.374 DEBUG:teuthology.orchestra.run.smithi072:> import os 2023-12-12T09:53:58.374 DEBUG:teuthology.orchestra.run.smithi072:> import stat 2023-12-12T09:53:58.374 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:53:58.374 DEBUG:teuthology.orchestra.run.smithi072:> print(os.stat("/home/ubuntu/cephtest/mnt.0/delete_me").st_ino) 2023-12-12T09:53:58.374 DEBUG:teuthology.orchestra.run.smithi072:> ' 2023-12-12T09:53:58.488 INFO:teuthology.orchestra.run.smithi072.stdout:1099511627776 2023-12-12T09:53:58.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-12T09:53:58.500 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.646+0000 7f830bdfaf40 1 Processor -- start 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.646+0000 7f830bdfaf40 1 -- start start 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830bdfaf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265277b0 0x560c265ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830bdfaf40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c2653da90 0x560c2653fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830bdfaf40 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c265cbce0 0x560c265400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830bdfaf40 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c264a2a80 con 0x560c265cbce0 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830bdfaf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c26459bc0 con 0x560c265277b0 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830bdfaf40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c2643c9c0 con 0x560c2653da90 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830a0fd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c265cbce0 0x560c265400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:58.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830a0fd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c265cbce0 0x560c265400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57480/0 (socket says 172.21.15.72:57480) 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830a0fd640 1 -- 172.21.15.72:0/3470169348 learned_addr learned my addr 172.21.15.72:0/3470169348 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f83090fb640 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c2653da90 0x560c2653fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f83098fc640 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265277b0 0x560c265ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830a0fd640 1 -- 172.21.15.72:0/3470169348 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c2653da90 msgr2=0x560c2653fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830a0fd640 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c2653da90 0x560c2653fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830a0fd640 1 -- 172.21.15.72:0/3470169348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265277b0 msgr2=0x560c265ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830a0fd640 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265277b0 0x560c265ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.647+0000 7f830a0fd640 1 -- 172.21.15.72:0/3470169348 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4005d20 con 0x560c265cbce0 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.648+0000 7f830a0fd640 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c265cbce0 0x560c265400e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8304001bf0 tx=0x7f830400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960d84a24ac9e3ef server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.648+0000 7f8303fff640 1 -- 172.21.15.72:0/3470169348 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830400e470 con 0x560c265cbce0 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.648+0000 7f8303fff640 1 -- 172.21.15.72:0/3470169348 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830400e5d0 con 0x560c265cbce0 2023-12-12T09:53:58.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.648+0000 7f8303fff640 1 -- 172.21.15.72:0/3470169348 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830400e9c0 con 0x560c265cbce0 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 -- 172.21.15.72:0/3470169348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c265cbce0 msgr2=0x560c265400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c265cbce0 0x560c265400e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8304001bf0 tx=0x7f830400b6c0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 -- 172.21.15.72:0/3470169348 shutdown_connections 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c265cbce0 0x560c265400e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c2653da90 0x560c2653fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 --2- 172.21.15.72:0/3470169348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265277b0 0x560c265ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 -- 172.21.15.72:0/3470169348 >> 172.21.15.72:0/3470169348 conn(0x560c2653e720 msgr2=0x560c265cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 -- 172.21.15.72:0/3470169348 shutdown_connections 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.649+0000 7f830bdfaf40 1 -- 172.21.15.72:0/3470169348 wait complete. 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.650+0000 7f830bdfaf40 1 Processor -- start 2023-12-12T09:53:58.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.650+0000 7f830bdfaf40 1 -- start start 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f830bdfaf40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c265277b0 0x560c2665e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f830bdfaf40 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c2653da90 0x560c26540e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f830bdfaf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265cbce0 0x560c265bb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f830bdfaf40 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c26430520 con 0x560c2653da90 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f830bdfaf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c26662230 con 0x560c265cbce0 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f830bdfaf40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c26661320 con 0x560c265277b0 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83090fb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c2653da90 0x560c26540e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83090fb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c2653da90 0x560c26540e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57496/0 (socket says 172.21.15.72:57496) 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83090fb640 1 -- 172.21.15.72:0/2003148167 learned_addr learned my addr 172.21.15.72:0/2003148167 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83090fb640 1 -- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c265277b0 msgr2=0x560c2665e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83098fc640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c265277b0 0x560c2665e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f830a0fd640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265cbce0 0x560c265bb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83090fb640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c265277b0 0x560c2665e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83090fb640 1 -- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265cbce0 msgr2=0x560c265bb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:58.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83090fb640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c265cbce0 0x560c265bb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.651+0000 7f83090fb640 1 -- 172.21.15.72:0/2003148167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2643b4a0 con 0x560c2653da90 2023-12-12T09:53:58.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.652+0000 7f83090fb640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c2653da90 0x560c26540e50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f82f4046d40 tx=0x7f82f4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:58.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.652+0000 7f8302ffd640 1 -- 172.21.15.72:0/2003148167 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f400a380 con 0x560c2653da90 2023-12-12T09:53:58.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.652+0000 7f8302ffd640 1 -- 172.21.15.72:0/2003148167 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40518e0 con 0x560c2653da90 2023-12-12T09:53:58.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.652+0000 7f8302ffd640 1 -- 172.21.15.72:0/2003148167 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4051c90 con 0x560c2653da90 2023-12-12T09:53:58.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.652+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c26448680 con 0x560c2653da90 2023-12-12T09:53:58.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.652+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c2643b830 con 0x560c2653da90 2023-12-12T09:53:58.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.654+0000 7f8302ffd640 1 -- 172.21.15.72:0/2003148167 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82f405a450 con 0x560c2653da90 2023-12-12T09:53:58.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.655+0000 7f8302ffd640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f82ec069820 0x7f82ec06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:58.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.655+0000 7f8302ffd640 1 -- 172.21.15.72:0/2003148167 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f82f40cc280 con 0x560c2653da90 2023-12-12T09:53:58.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.655+0000 7f83098fc640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f82ec069820 0x7f82ec06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:58.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.655+0000 7f830bdfaf40 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x560c265bc510 0x560c26664ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:58.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.655+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 --> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x560c266653e0 con 0x560c265bc510 2023-12-12T09:53:58.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.656+0000 7f83098fc640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f82ec069820 0x7f82ec06bce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x560c265bdc30 tx=0x7f82fc007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:58.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.656+0000 7f830a0fd640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x560c265bc510 0x560c26664ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:58.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.657+0000 7f830a0fd640 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x560c265bc510 0x560c26664ea0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:58.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830a0fd640 1 -- 172.21.15.72:0/2003148167 <== osd.7 v2:172.21.15.113:6824/1904806807 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x7f8304014070 con 0x560c265bc510 2023-12-12T09:53:58.660 INFO:teuthology.orchestra.run.smithi072.stdout:cephfs_metadata/10000000000.00000000 mtime 2023-12-12T09:53:58.000000+0000, size 0 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x560c265bc510 msgr2=0x560c26664ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x560c265bc510 0x560c26664ea0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f82ec069820 msgr2=0x7f82ec06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f82ec069820 0x7f82ec06bce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x560c265bdc30 tx=0x7f82fc007040 comp rx=0 tx=0).stop 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c2653da90 msgr2=0x560c26540e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 --2- 172.21.15.72:0/2003148167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560c2653da90 0x560c26540e50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f82f4046d40 tx=0x7f82f4002fb0 comp rx=0 tx=0).stop 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830a0fd640 1 -- 172.21.15.72:0/2003148167 reap_dead start 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 shutdown_connections 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 >> 172.21.15.72:0/2003148167 conn(0x560c2653e720 msgr2=0x560c265ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 shutdown_connections 2023-12-12T09:53:58.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:53:58.658+0000 7f830bdfaf40 1 -- 172.21.15.72:0/2003148167 wait complete. 2023-12-12T09:53:58.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:53:58.664 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf delete_me) 2023-12-12T09:54:02.780 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:03.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.004+0000 7f68189a0640 1 Processor -- start 2023-12-12T09:54:03.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.004+0000 7f68189a0640 1 -- start start 2023-12-12T09:54:03.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.006+0000 7f68189a0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140b1cb0 0x7f68140b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:03.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.006+0000 7f68189a0640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68140cc400 con 0x7f68140cf5a0 2023-12-12T09:54:03.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.006+0000 7f68189a0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68140cc6e0 con 0x7f681415e640 2023-12-12T09:54:03.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.006+0000 7f68189a0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68140ccca0 con 0x7f68140b1cb0 2023-12-12T09:54:03.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.007+0000 7f6812ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140b1cb0 0x7f68140b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:03.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.007+0000 7f6812ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140b1cb0 0x7f68140b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:56120/0 (socket says 172.21.15.72:56120) 2023-12-12T09:54:03.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.007+0000 7f6812ffd640 1 -- 172.21.15.72:0/1833497241 learned_addr learned my addr 172.21.15.72:0/1833497241 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:03.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.007+0000 7f68137fe640 1 --1- 172.21.15.72:0/1833497241 >> v1:172.21.15.113:6789/0 conn(0x7f681415e640 0x7f68140cee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:60914/0 (socket says 172.21.15.72:60914) 2023-12-12T09:54:03.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.007+0000 7f6812ffd640 1 -- 172.21.15.72:0/1833497241 >> v1:172.21.15.113:6789/0 conn(0x7f681415e640 legacy=0x7f68140cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:03.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.007+0000 7f6812ffd640 1 -- 172.21.15.72:0/1833497241 >> v1:172.21.15.72:6789/0 conn(0x7f68140cf5a0 legacy=0x7f681415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:03.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.007+0000 7f6812ffd640 1 -- 172.21.15.72:0/1833497241 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140cc9c0 con 0x7f68140b1cb0 2023-12-12T09:54:03.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.010+0000 7f6812ffd640 1 --2- 172.21.15.72:0/1833497241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140b1cb0 0x7f68140b3b00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f68000098a0 tx=0x7f680002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d658385d1a5e621 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:03.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.011+0000 7f68127fc640 1 -- 172.21.15.72:0/1833497241 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800030840 con 0x7f68140b1cb0 2023-12-12T09:54:03.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.011+0000 7f68127fc640 1 -- 172.21.15.72:0/1833497241 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68000309a0 con 0x7f68140b1cb0 2023-12-12T09:54:03.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.011+0000 7f68127fc640 1 -- 172.21.15.72:0/1833497241 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800030d30 con 0x7f68140b1cb0 2023-12-12T09:54:03.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.012+0000 7f68189a0640 1 -- 172.21.15.72:0/1833497241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140b1cb0 msgr2=0x7f68140b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:03.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.012+0000 7f68189a0640 1 --2- 172.21.15.72:0/1833497241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140b1cb0 0x7f68140b3b00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f68000098a0 tx=0x7f680002b980 comp rx=0 tx=0).stop 2023-12-12T09:54:03.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.013+0000 7f68189a0640 1 -- 172.21.15.72:0/1833497241 shutdown_connections 2023-12-12T09:54:03.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.013+0000 7f68189a0640 1 --2- 172.21.15.72:0/1833497241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140b1cb0 0x7f68140b3b00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.013+0000 7f68189a0640 1 -- 172.21.15.72:0/1833497241 >> 172.21.15.72:0/1833497241 conn(0x7f68140b2800 msgr2=0x7f68140b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:03.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.013+0000 7f68189a0640 1 -- 172.21.15.72:0/1833497241 shutdown_connections 2023-12-12T09:54:03.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.013+0000 7f68189a0640 1 -- 172.21.15.72:0/1833497241 wait complete. 2023-12-12T09:54:03.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.016+0000 7f68189a0640 1 Processor -- start 2023-12-12T09:54:03.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.016+0000 7f68189a0640 1 -- start start 2023-12-12T09:54:03.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68189a0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68140b1cb0 0x7f6814143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:03.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68189a0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140cf5a0 0x7f68141f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:03.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68189a0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681415e640 0x7f68140c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68189a0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814159ce0 con 0x7f68140b1cb0 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68189a0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814159340 con 0x7f681415e640 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68189a0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6814159620 con 0x7f68140cf5a0 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68137fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68140b1cb0 0x7f6814143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68137fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68140b1cb0 0x7f6814143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57502/0 (socket says 172.21.15.72:57502) 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.018+0000 7f68137fe640 1 -- 172.21.15.72:0/419681987 learned_addr learned my addr 172.21.15.72:0/419681987 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.019+0000 7f6813fff640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681415e640 0x7f68140c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.019+0000 7f6812ffd640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140cf5a0 0x7f68141f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:03.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.020+0000 7f68137fe640 1 -- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140cf5a0 msgr2=0x7f68141f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:03.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.020+0000 7f68137fe640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140cf5a0 0x7f68141f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.020+0000 7f68137fe640 1 -- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681415e640 msgr2=0x7f68140c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:03.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.020+0000 7f68137fe640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681415e640 0x7f68140c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.020+0000 7f68137fe640 1 -- 172.21.15.72:0/419681987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000094d0 con 0x7f68140b1cb0 2023-12-12T09:54:03.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.020+0000 7f6812ffd640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140cf5a0 0x7f68141f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:03.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.020+0000 7f68137fe640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68140b1cb0 0x7f6814143890 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6804001e90 tx=0x7f680400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:03.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.021+0000 7f680bfff640 1 -- 172.21.15.72:0/419681987 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804007480 con 0x7f68140b1cb0 2023-12-12T09:54:03.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.021+0000 7f680bfff640 1 -- 172.21.15.72:0/419681987 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68040075e0 con 0x7f68140b1cb0 2023-12-12T09:54:03.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.021+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681415eba0 con 0x7f68140b1cb0 2023-12-12T09:54:03.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.021+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68141f14f0 con 0x7f68140b1cb0 2023-12-12T09:54:03.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.022+0000 7f680bfff640 1 -- 172.21.15.72:0/419681987 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040049e0 con 0x7f68140b1cb0 2023-12-12T09:54:03.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.023+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f0005250 con 0x7f68140b1cb0 2023-12-12T09:54:03.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.025+0000 7f680bfff640 1 -- 172.21.15.72:0/419681987 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6804004b40 con 0x7f68140b1cb0 2023-12-12T09:54:03.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.027+0000 7f680bfff640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67fc06ae90 0x7f67fc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:03.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.027+0000 7f680bfff640 1 -- 172.21.15.72:0/419681987 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f680408d1a0 con 0x7f68140b1cb0 2023-12-12T09:54:03.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.033+0000 7f6812ffd640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67fc06ae90 0x7f67fc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:03.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.034+0000 7f6812ffd640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67fc06ae90 0x7f67fc06d350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6800036000 tx=0x7f680003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:03.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.038+0000 7f680bfff640 1 -- 172.21.15.72:0/419681987 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6804058260 con 0x7f68140b1cb0 2023-12-12T09:54:03.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.333+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67f0005500 con 0x7f68140b1cb0 2023-12-12T09:54:03.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.334+0000 7f680bfff640 1 -- 172.21.15.72:0/419681987 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f680405b880 con 0x7f68140b1cb0 2023-12-12T09:54:03.336 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:03.336 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:03.336 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67fc06ae90 msgr2=0x7f67fc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67fc06ae90 0x7f67fc06d350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6800036000 tx=0x7f680003b040 comp rx=0 tx=0).stop 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68140b1cb0 msgr2=0x7f6814143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68140b1cb0 0x7f6814143890 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6804001e90 tx=0x7f680400c960 comp rx=0 tx=0).stop 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 shutdown_connections 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f67fc06ae90 0x7f67fc06d350 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681415e640 0x7f68140c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68140cf5a0 0x7f68141f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 --2- 172.21.15.72:0/419681987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68140b1cb0 0x7f6814143890 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 >> 172.21.15.72:0/419681987 conn(0x7f68140b2800 msgr2=0x7f68140b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 shutdown_connections 2023-12-12T09:54:03.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.337+0000 7f68189a0640 1 -- 172.21.15.72:0/419681987 wait complete. 2023-12-12T09:54:03.358 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-12-12T09:54:03.398 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:54:03.407 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:54:03.799 INFO:teuthology.orchestra.run.smithi113.stdout:{"message":"","return_code":0} 2023-12-12T09:54:03.799 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-12-12T09:54:03.800 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:03.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.989+0000 7fe932a87640 1 Processor -- start 2023-12-12T09:54:03.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.990+0000 7fe932a87640 1 -- start start 2023-12-12T09:54:03.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.991+0000 7fe932a87640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe92c143e00 0x7fe92c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:03.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.991+0000 7fe932a87640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe92c072180 0x7fe92c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:03.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.991+0000 7fe932a87640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c06c410 0x7fe92c0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:03.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.991+0000 7fe932a87640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0b3bb0 con 0x7fe92c143e00 2023-12-12T09:54:03.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.991+0000 7fe932a87640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c1441e0 con 0x7fe92c072180 2023-12-12T09:54:03.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.991+0000 7fe932a87640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c072680 con 0x7fe92c06c410 2023-12-12T09:54:03.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.991+0000 7fe930823640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe92c143e00 0x7fe92c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:03.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe930823640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe92c143e00 0x7fe92c0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57518/0 (socket says 172.21.15.72:57518) 2023-12-12T09:54:03.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe930823640 1 -- 172.21.15.72:0/1424613072 learned_addr learned my addr 172.21.15.72:0/1424613072 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe92bfff640 1 --2- 172.21.15.72:0/1424613072 >> v2:172.21.15.113:3300/0 conn(0x7fe92c072180 0x7fe92c0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe931024640 1 --2- 172.21.15.72:0/1424613072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c06c410 0x7fe92c0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe930823640 1 -- 172.21.15.72:0/1424613072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c06c410 msgr2=0x7fe92c0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe930823640 1 --2- 172.21.15.72:0/1424613072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c06c410 0x7fe92c0b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe930823640 1 -- 172.21.15.72:0/1424613072 >> v2:172.21.15.113:3300/0 conn(0x7fe92c072180 msgr2=0x7fe92c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe930823640 1 --2- 172.21.15.72:0/1424613072 >> v2:172.21.15.113:3300/0 conn(0x7fe92c072180 0x7fe92c0b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe930823640 1 -- 172.21.15.72:0/1424613072 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0b1e30 con 0x7fe92c143e00 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe92bfff640 1 --2- 172.21.15.72:0/1424613072 >> v2:172.21.15.113:3300/0 conn(0x7fe92c072180 0x7fe92c0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.992+0000 7fe931024640 1 --2- 172.21.15.72:0/1424613072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c06c410 0x7fe92c0b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.994+0000 7fe930823640 1 --2- 172.21.15.72:0/1424613072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c0b4040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe92c067da0 tx=0x7fe92002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaebdf181b8dcb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:03.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.994+0000 7fe92b7fe640 1 -- 172.21.15.72:0/1424613072 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920031870 con 0x7fe92c143e00 2023-12-12T09:54:03.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.994+0000 7fe92b7fe640 1 -- 172.21.15.72:0/1424613072 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9200319d0 con 0x7fe92c143e00 2023-12-12T09:54:03.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.994+0000 7fe92b7fe640 1 -- 172.21.15.72:0/1424613072 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920031d60 con 0x7fe92c143e00 2023-12-12T09:54:03.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 -- 172.21.15.72:0/1424613072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 msgr2=0x7fe92c0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:03.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 --2- 172.21.15.72:0/1424613072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c0b4040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe92c067da0 tx=0x7fe92002b910 comp rx=0 tx=0).stop 2023-12-12T09:54:03.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 -- 172.21.15.72:0/1424613072 shutdown_connections 2023-12-12T09:54:03.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 --2- 172.21.15.72:0/1424613072 >> v2:172.21.15.113:3300/0 conn(0x7fe92c072180 0x7fe92c0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 --2- 172.21.15.72:0/1424613072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c06c410 0x7fe92c0b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 --2- 172.21.15.72:0/1424613072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c0b4040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:03.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 -- 172.21.15.72:0/1424613072 >> 172.21.15.72:0/1424613072 conn(0x7fe92c0b0a90 msgr2=0x7fe92c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:03.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 -- 172.21.15.72:0/1424613072 shutdown_connections 2023-12-12T09:54:03.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.995+0000 7fe932a87640 1 -- 172.21.15.72:0/1424613072 wait complete. 2023-12-12T09:54:03.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.997+0000 7fe932a87640 1 Processor -- start 2023-12-12T09:54:03.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.997+0000 7fe932a87640 1 -- start start 2023-12-12T09:54:04.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe932a87640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92c06c410 0x7fe92c0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe932a87640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c072180 0x7fe92c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe932a87640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c1f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe932a87640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c15d910 con 0x7fe92c143e00 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe932a87640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0ae5a0 con 0x7fe92c06c410 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe932a87640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c1f4cf0 con 0x7fe92c072180 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe931024640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c1f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe931024640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c1f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57532/0 (socket says 172.21.15.72:57532) 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe931024640 1 -- 172.21.15.72:0/1471389772 learned_addr learned my addr 172.21.15.72:0/1471389772 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.998+0000 7fe930823640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92c06c410 0x7fe92c0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe92bfff640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c072180 0x7fe92c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe931024640 1 -- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c072180 msgr2=0x7fe92c15e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe931024640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c072180 0x7fe92c15e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe931024640 1 -- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92c06c410 msgr2=0x7fe92c0ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe931024640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92c06c410 0x7fe92c0ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe931024640 1 -- 172.21.15.72:0/1471389772 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9200094d0 con 0x7fe92c143e00 2023-12-12T09:54:04.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe930823640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92c06c410 0x7fe92c0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:54:04.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe931024640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c1f4250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe92400ce20 tx=0x7fe92400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:04.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe9297fa640 1 -- 172.21.15.72:0/1471389772 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924003b90 con 0x7fe92c143e00 2023-12-12T09:54:04.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe9297fa640 1 -- 172.21.15.72:0/1471389772 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe924003d10 con 0x7fe92c143e00 2023-12-12T09:54:04.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe9297fa640 1 -- 172.21.15.72:0/1471389772 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924004260 con 0x7fe92c143e00 2023-12-12T09:54:04.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c1f4fd0 con 0x7fe92c143e00 2023-12-12T09:54:04.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:03.999+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c15de00 con 0x7fe92c143e00 2023-12-12T09:54:04.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.001+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c1f51c0 con 0x7fe92c143e00 2023-12-12T09:54:04.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.001+0000 7fe9297fa640 1 -- 172.21.15.72:0/1471389772 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9240043c0 con 0x7fe92c143e00 2023-12-12T09:54:04.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.002+0000 7fe9297fa640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe91006ae40 0x7fe91006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.002+0000 7fe9297fa640 1 -- 172.21.15.72:0/1471389772 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe92408a080 con 0x7fe92c143e00 2023-12-12T09:54:04.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.003+0000 7fe930823640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe91006ae40 0x7fe91006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.003+0000 7fe930823640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe91006ae40 0x7fe91006d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe920036000 tx=0x7fe92003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:04.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.007+0000 7fe9297fa640 1 -- 172.21.15.72:0/1471389772 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9240bce90 con 0x7fe92c143e00 2023-12-12T09:54:04.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.233+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe92c15e020 con 0x7fe92c143e00 2023-12-12T09:54:04.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.234+0000 7fe9297fa640 1 -- 172.21.15.72:0/1471389772 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fe924054f30 con 0x7fe92c143e00 2023-12-12T09:54:04.236 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:04.236 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:04.236 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe91006ae40 msgr2=0x7fe91006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe91006ae40 0x7fe91006d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe920036000 tx=0x7fe92003c040 comp rx=0 tx=0).stop 2023-12-12T09:54:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 msgr2=0x7fe92c1f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c1f4250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe92400ce20 tx=0x7fe92400a710 comp rx=0 tx=0).stop 2023-12-12T09:54:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 shutdown_connections 2023-12-12T09:54:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe91006ae40 0x7fe91006d300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe92c143e00 0x7fe92c1f4250 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92c072180 0x7fe92c15e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 --2- 172.21.15.72:0/1471389772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92c06c410 0x7fe92c0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 >> 172.21.15.72:0/1471389772 conn(0x7fe92c0b0a90 msgr2=0x7fe92c142660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:04.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 shutdown_connections 2023-12-12T09:54:04.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.237+0000 7fe932a87640 1 -- 172.21.15.72:0/1471389772 wait complete. 2023-12-12T09:54:04.257 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json perf dump mds_cache strays_created 2023-12-12T09:54:04.497 INFO:teuthology.orchestra.run.smithi113.stdout:{"mds_cache":{"strays_created":1001}} 2023-12-12T09:54:04.498 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_cache": { "strays_created": 1001 } } 2023-12-12T09:54:04.498 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:04.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.716+0000 7f0303939640 1 Processor -- start 2023-12-12T09:54:04.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.716+0000 7f0303939640 1 -- start start 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.717+0000 7f0303939640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc143e00 0x7f02fc0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.717+0000 7f0303939640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02fc072180 0x7f02fc0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.717+0000 7f0303939640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f02fc06c410 0x7f02fc0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0303939640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02fc0b3bb0 con 0x7f02fc06c410 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0303939640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02fc1441e0 con 0x7f02fc072180 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0303939640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc072680 con 0x7f02fc143e00 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0301ed6640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f02fc06c410 0x7f02fc0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0301ed6640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f02fc06c410 0x7f02fc0b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57540/0 (socket says 172.21.15.72:57540) 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0301ed6640 1 -- 172.21.15.72:0/483089412 learned_addr learned my addr 172.21.15.72:0/483089412 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0300ed4640 1 --2- 172.21.15.72:0/483089412 >> v2:172.21.15.113:3300/0 conn(0x7f02fc072180 0x7f02fc0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f03016d5640 1 --2- 172.21.15.72:0/483089412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc143e00 0x7f02fc0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0301ed6640 1 -- 172.21.15.72:0/483089412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc143e00 msgr2=0x7f02fc0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:04.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0301ed6640 1 --2- 172.21.15.72:0/483089412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc143e00 0x7f02fc0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0301ed6640 1 -- 172.21.15.72:0/483089412 >> v2:172.21.15.113:3300/0 conn(0x7f02fc072180 msgr2=0x7f02fc0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:04.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0301ed6640 1 --2- 172.21.15.72:0/483089412 >> v2:172.21.15.113:3300/0 conn(0x7f02fc072180 0x7f02fc0b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.718+0000 7f0301ed6640 1 -- 172.21.15.72:0/483089412 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc0b1e30 con 0x7f02fc06c410 2023-12-12T09:54:04.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.720+0000 7f0301ed6640 1 --2- 172.21.15.72:0/483089412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc06c410 0x7f02fc0b4ac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02fc06b810 tx=0x7f02f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76fe0a6464eecfb server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:04.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.720+0000 7f02f3fff640 1 -- 172.21.15.72:0/483089412 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f402ca70 con 0x7f02fc06c410 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.720+0000 7f02f3fff640 1 -- 172.21.15.72:0/483089412 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f402cbd0 con 0x7f02fc06c410 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.720+0000 7f02f3fff640 1 -- 172.21.15.72:0/483089412 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f40343f0 con 0x7f02fc06c410 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 -- 172.21.15.72:0/483089412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc06c410 msgr2=0x7f02fc0b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 --2- 172.21.15.72:0/483089412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc06c410 0x7f02fc0b4ac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02fc06b810 tx=0x7f02f402bed0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 -- 172.21.15.72:0/483089412 shutdown_connections 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 --2- 172.21.15.72:0/483089412 >> v2:172.21.15.113:3300/0 conn(0x7f02fc072180 0x7f02fc0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 --2- 172.21.15.72:0/483089412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc06c410 0x7f02fc0b4ac0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 --2- 172.21.15.72:0/483089412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc143e00 0x7f02fc0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 -- 172.21.15.72:0/483089412 >> 172.21.15.72:0/483089412 conn(0x7f02fc0b0a90 msgr2=0x7f02fc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 -- 172.21.15.72:0/483089412 shutdown_connections 2023-12-12T09:54:04.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.721+0000 7f0303939640 1 -- 172.21.15.72:0/483089412 wait complete. 2023-12-12T09:54:04.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.723+0000 7f0303939640 1 Processor -- start 2023-12-12T09:54:04.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.723+0000 7f0303939640 1 -- start start 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0303939640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02fc06c410 0x7f02fc13e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0303939640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc072180 0x7f02fc13cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0303939640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc143e00 0x7f02fc13d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0303939640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1f3f80 con 0x7f02fc143e00 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0303939640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc13d760 con 0x7f02fc06c410 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0303939640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc140bf0 con 0x7f02fc072180 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0301ed6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc143e00 0x7f02fc13d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0301ed6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc143e00 0x7f02fc13d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57554/0 (socket says 172.21.15.72:57554) 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0301ed6640 1 -- 172.21.15.72:0/2678375819 learned_addr learned my addr 172.21.15.72:0/2678375819 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:04.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f03016d5640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02fc06c410 0x7f02fc13e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0300ed4640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc072180 0x7f02fc13cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0301ed6640 1 -- 172.21.15.72:0/2678375819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc072180 msgr2=0x7f02fc13cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0301ed6640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc072180 0x7f02fc13cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0301ed6640 1 -- 172.21.15.72:0/2678375819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02fc06c410 msgr2=0x7f02fc13e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.724+0000 7f0301ed6640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02fc06c410 0x7f02fc13e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.725+0000 7f0301ed6640 1 -- 172.21.15.72:0/2678375819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f4009b30 con 0x7f02fc143e00 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.725+0000 7f0301ed6640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc143e00 0x7f02fc13d220 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f02f4036040 tx=0x7f02f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.725+0000 7f02f1ffb640 1 -- 172.21.15.72:0/2678375819 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4009510 con 0x7f02fc143e00 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.725+0000 7f02f1ffb640 1 -- 172.21.15.72:0/2678375819 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f403e450 con 0x7f02fc143e00 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.725+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02fc140ed0 con 0x7f02fc143e00 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.725+0000 7f02f1ffb640 1 -- 172.21.15.72:0/2678375819 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f403e7e0 con 0x7f02fc143e00 2023-12-12T09:54:04.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.725+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc141490 con 0x7f02fc143e00 2023-12-12T09:54:04.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.726+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02fc141680 con 0x7f02fc143e00 2023-12-12T09:54:04.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.727+0000 7f02f1ffb640 1 -- 172.21.15.72:0/2678375819 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02f403e940 con 0x7f02fc143e00 2023-12-12T09:54:04.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.728+0000 7f02f1ffb640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02e006ae70 0x7f02e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:04.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.729+0000 7f02f1ffb640 1 -- 172.21.15.72:0/2678375819 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f02f40ac2d0 con 0x7f02fc143e00 2023-12-12T09:54:04.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.729+0000 7f03016d5640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02e006ae70 0x7f02e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:04.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.730+0000 7f03016d5640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02e006ae70 0x7f02e006d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f02f8006fd0 tx=0x7f02f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:04.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.733+0000 7f02f1ffb640 1 -- 172.21.15.72:0/2678375819 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02f4077210 con 0x7f02fc143e00 2023-12-12T09:54:04.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.962+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02fc0745d0 con 0x7f02fc143e00 2023-12-12T09:54:04.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.963+0000 7f02f1ffb640 1 -- 172.21.15.72:0/2678375819 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f02f4047e90 con 0x7f02fc143e00 2023-12-12T09:54:04.965 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:04.965 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:04.965 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:04.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02e006ae70 msgr2=0x7f02e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02e006ae70 0x7f02e006d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f02f8006fd0 tx=0x7f02f8008040 comp rx=0 tx=0).stop 2023-12-12T09:54:04.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc143e00 msgr2=0x7f02fc13d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:04.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc143e00 0x7f02fc13d220 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f02f4036040 tx=0x7f02f4004a40 comp rx=0 tx=0).stop 2023-12-12T09:54:04.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 shutdown_connections 2023-12-12T09:54:04.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02e006ae70 0x7f02e006d330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02fc143e00 0x7f02fc13d220 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02fc072180 0x7f02fc13cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 --2- 172.21.15.72:0/2678375819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02fc06c410 0x7f02fc13e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:04.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 >> 172.21.15.72:0/2678375819 conn(0x7f02fc0b0a90 msgr2=0x7f02fc06d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:04.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 shutdown_connections 2023-12-12T09:54:04.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:04.966+0000 7f0303939640 1 -- 172.21.15.72:0/2678375819 wait complete. 2023-12-12T09:54:04.986 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json perf dump mds_cache strays_enqueued 2023-12-12T09:54:05.227 INFO:teuthology.orchestra.run.smithi113.stdout:{"mds_cache":{"strays_enqueued":942}} 2023-12-12T09:54:05.227 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_cache": { "strays_enqueued": 942 } } 2023-12-12T09:54:05.227 DEBUG:tasks.ceph_test_case:wait_until_equal: 942 != 1001, waiting (timeout=600)... 2023-12-12T09:54:10.228 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:10.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.451+0000 7f7df4e56640 1 Processor -- start 2023-12-12T09:54:10.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.451+0000 7f7df4e56640 1 -- start start 2023-12-12T09:54:10.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.453+0000 7f7df4e56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df00b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.453+0000 7f7df4e56640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7df006c410 0x7f7df015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.453+0000 7f7df4e56640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df00b3bb0 con 0x7f7df0143e00 2023-12-12T09:54:10.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.453+0000 7f7df4e56640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df01441e0 con 0x7f7df006c410 2023-12-12T09:54:10.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.453+0000 7f7df4e56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0072680 con 0x7f7df0072180 2023-12-12T09:54:10.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.453+0000 7f7deed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7df006c410 0x7f7df015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.453+0000 7f7dedd9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df00b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.454+0000 7f7deed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7df006c410 0x7f7df015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:37074/0 (socket says 172.21.15.72:37074) 2023-12-12T09:54:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.454+0000 7f7deed9d640 1 -- 172.21.15.72:0/1364511244 learned_addr learned my addr 172.21.15.72:0/1364511244 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.454+0000 7f7dedd9b640 1 -- 172.21.15.72:0/1364511244 >> v2:172.21.15.113:3300/0 conn(0x7f7df006c410 msgr2=0x7f7df015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.454+0000 7f7dedd9b640 1 --2- 172.21.15.72:0/1364511244 >> v2:172.21.15.113:3300/0 conn(0x7f7df006c410 0x7f7df015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.454+0000 7f7dedd9b640 1 -- 172.21.15.72:0/1364511244 >> v1:172.21.15.113:6789/0 conn(0x7f7df0143e00 legacy=0x7f7df00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.454+0000 7f7dedd9b640 1 -- 172.21.15.72:0/1364511244 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00b1e30 con 0x7f7df0072180 2023-12-12T09:54:10.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.456+0000 7f7dedd9b640 1 --2- 172.21.15.72:0/1364511244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df00b4750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7de40097a0 tx=0x7f7de402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9117a9870c997665 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:10.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.456+0000 7f7ded59a640 1 -- 172.21.15.72:0/1364511244 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4031830 con 0x7f7df0072180 2023-12-12T09:54:10.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.456+0000 7f7ded59a640 1 -- 172.21.15.72:0/1364511244 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de4031990 con 0x7f7df0072180 2023-12-12T09:54:10.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.456+0000 7f7ded59a640 1 -- 172.21.15.72:0/1364511244 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4031d20 con 0x7f7df0072180 2023-12-12T09:54:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.457+0000 7f7df4e56640 1 -- 172.21.15.72:0/1364511244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 msgr2=0x7f7df00b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.457+0000 7f7df4e56640 1 --2- 172.21.15.72:0/1364511244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df00b4750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7de40097a0 tx=0x7f7de402b8d0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.457+0000 7f7df4e56640 1 -- 172.21.15.72:0/1364511244 shutdown_connections 2023-12-12T09:54:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.457+0000 7f7df4e56640 1 --2- 172.21.15.72:0/1364511244 >> v2:172.21.15.113:3300/0 conn(0x7f7df006c410 0x7f7df015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.457+0000 7f7df4e56640 1 --2- 172.21.15.72:0/1364511244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df00b4750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.457+0000 7f7df4e56640 1 -- 172.21.15.72:0/1364511244 >> 172.21.15.72:0/1364511244 conn(0x7f7df00b0a90 msgr2=0x7f7df00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.457+0000 7f7df4e56640 1 -- 172.21.15.72:0/1364511244 shutdown_connections 2023-12-12T09:54:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.457+0000 7f7df4e56640 1 -- 172.21.15.72:0/1364511244 wait complete. 2023-12-12T09:54:10.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.459+0000 7f7df4e56640 1 Processor -- start 2023-12-12T09:54:10.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.459+0000 7f7df4e56640 1 -- start start 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7df4e56640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7df006c410 0x7f7df00afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7df4e56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df013ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7df4e56640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7df0143e00 0x7f7df013b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7df4e56640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df0137230 con 0x7f7df006c410 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7df4e56640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00aeb50 con 0x7f7df0143e00 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7df4e56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00aee30 con 0x7f7df0072180 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7dee59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7df006c410 0x7f7df00afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7dee59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7df006c410 0x7f7df00afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46634/0 (socket says 172.21.15.72:46634) 2023-12-12T09:54:10.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7dee59c640 1 -- 172.21.15.72:0/3972215657 learned_addr learned my addr 172.21.15.72:0/3972215657 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7dedd9b640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df013ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7dee59c640 1 -- 172.21.15.72:0/3972215657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 msgr2=0x7f7df013ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7dee59c640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df013ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7dee59c640 1 -- 172.21.15.72:0/3972215657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7df0143e00 msgr2=0x7f7df013b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7deed9d640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7df0143e00 0x7f7df013b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.460+0000 7f7dee59c640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7df0143e00 0x7f7df013b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.461+0000 7f7dee59c640 1 -- 172.21.15.72:0/3972215657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4009400 con 0x7f7df006c410 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.461+0000 7f7dee59c640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7df006c410 0x7f7df00afbf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7de000c400 tx=0x7f7de000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.461+0000 7f7ddeffd640 1 -- 172.21.15.72:0/3972215657 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de00052d0 con 0x7f7df006c410 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.461+0000 7f7ddeffd640 1 -- 172.21.15.72:0/3972215657 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de0005430 con 0x7f7df006c410 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.461+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df00af110 con 0x7f7df006c410 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.461+0000 7f7ddeffd640 1 -- 172.21.15.72:0/3972215657 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0003830 con 0x7f7df006c410 2023-12-12T09:54:10.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.461+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df00af6d0 con 0x7f7df006c410 2023-12-12T09:54:10.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.463+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df00745d0 con 0x7f7df006c410 2023-12-12T09:54:10.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.463+0000 7f7ddeffd640 1 -- 172.21.15.72:0/3972215657 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7de0003990 con 0x7f7df006c410 2023-12-12T09:54:10.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.464+0000 7f7ddeffd640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7dd006ae40 0x7f7dd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.464+0000 7f7ddeffd640 1 -- 172.21.15.72:0/3972215657 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7de0088cd0 con 0x7f7df006c410 2023-12-12T09:54:10.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.465+0000 7f7dedd9b640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7dd006ae40 0x7f7dd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.465+0000 7f7dedd9b640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7dd006ae40 0x7f7dd006d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7de4036000 tx=0x7f7de4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:10.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.470+0000 7f7ddeffd640 1 -- 172.21.15.72:0/3972215657 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7de0053c20 con 0x7f7df006c410 2023-12-12T09:54:10.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.700+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7df006e020 con 0x7f7df006c410 2023-12-12T09:54:10.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.702+0000 7f7ddeffd640 1 -- 172.21.15.72:0/3972215657 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f7de0017090 con 0x7f7df006c410 2023-12-12T09:54:10.704 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:10.704 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:10.704 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:10.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.704+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7dd006ae40 msgr2=0x7f7dd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.704+0000 7f7df4e56640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7dd006ae40 0x7f7dd006d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7de4036000 tx=0x7f7de4008030 comp rx=0 tx=0).stop 2023-12-12T09:54:10.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.704+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7df006c410 msgr2=0x7f7df00afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.704+0000 7f7df4e56640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7df006c410 0x7f7df00afbf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7de000c400 tx=0x7f7de000c8c0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.705+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 shutdown_connections 2023-12-12T09:54:10.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.705+0000 7f7df4e56640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7dd006ae40 0x7f7dd006d300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.705+0000 7f7df4e56640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7df0143e00 0x7f7df013b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.705+0000 7f7df4e56640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7df0072180 0x7f7df013ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.705+0000 7f7df4e56640 1 --2- 172.21.15.72:0/3972215657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7df006c410 0x7f7df00afbf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.705+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 >> 172.21.15.72:0/3972215657 conn(0x7f7df00b0a90 msgr2=0x7f7df0070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:10.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.705+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 shutdown_connections 2023-12-12T09:54:10.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:10.705+0000 7f7df4e56640 1 -- 172.21.15.72:0/3972215657 wait complete. 2023-12-12T09:54:10.724 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json perf dump mds_cache strays_enqueued 2023-12-12T09:54:10.964 INFO:teuthology.orchestra.run.smithi113.stdout:{"mds_cache":{"strays_enqueued":1001}} 2023-12-12T09:54:10.965 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_cache": { "strays_enqueued": 1001 } } 2023-12-12T09:54:10.965 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:11.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.186+0000 7fee6cf53640 1 Processor -- start 2023-12-12T09:54:11.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.186+0000 7fee6cf53640 1 -- start start 2023-12-12T09:54:11.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.187+0000 7fee6cf53640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 0x7fee6815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.188+0000 7fee6cf53640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6806ca40 con 0x7fee6815e0e0 2023-12-12T09:54:11.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.188+0000 7fee6cf53640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee68071cd0 con 0x7fee68157a60 2023-12-12T09:54:11.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.188+0000 7fee6cf53640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815e550 con 0x7fee6806c410 2023-12-12T09:54:11.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.188+0000 7fee66d9d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fee6815e0e0 0x7fee680cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:44918/0 (socket says 172.21.15.72:44918) 2023-12-12T09:54:11.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.188+0000 7fee66d9d640 1 -- 172.21.15.72:0/934249508 learned_addr learned my addr 172.21.15.72:0/934249508 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:11.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.188+0000 7fee5ffff640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 0x7fee6815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.188+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991800951 0 0) 0x7fee6806ca40 con 0x7fee6815e0e0 2023-12-12T09:54:11.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.188+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee54005730 con 0x7fee6815e0e0 2023-12-12T09:54:11.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198632014 0 0) 0x7fee54005730 con 0x7fee6815e0e0 2023-12-12T09:54:11.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 msgr2=0x7fee6815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:11.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 0x7fee6815dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 >> v1:172.21.15.113:6789/0 conn(0x7fee68157a60 legacy=0x7fee6806de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68070f10 con 0x7fee6815e0e0 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fee600019f0 con 0x7fee6815e0e0 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 >> v1:172.21.15.72:6789/0 conn(0x7fee6815e0e0 legacy=0x7fee680cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 0x7fee54005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee54006130 0x7fee540069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006f10 0x7fee54007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815e550 con 0x7fee54006130 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5400dd20 con 0x7fee54006f10 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5400dee0 con 0x7fee6806c410 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.189+0000 7fee5ffff640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee54006130 0x7fee540069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee5ffff640 1 -- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 msgr2=0x7fee54005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee66d9d640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006f10 0x7fee54007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee6659c640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 0x7fee54005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee5ffff640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 0x7fee54005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee5ffff640 1 -- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006f10 msgr2=0x7fee54007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee5ffff640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006f10 0x7fee54007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee5ffff640 1 -- 172.21.15.72:0/934249508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54005730 con 0x7fee54006130 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee66d9d640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006f10 0x7fee54007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.190+0000 7fee6659c640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee6806c410 0x7fee54005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.191+0000 7fee5ffff640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee54006130 0x7fee540069d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fee50009ac0 tx=0x7fee54007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6ba9720ed65a78 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.191+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5002c6b0 con 0x7fee54006130 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.191+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5002c810 con 0x7fee54006130 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.191+0000 7fee65d9b640 1 -- 172.21.15.72:0/934249508 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5002cba0 con 0x7fee54006130 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.192+0000 7fee6cf53640 1 -- 172.21.15.72:0/934249508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee54006130 msgr2=0x7fee540069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.192+0000 7fee6cf53640 1 --2- 172.21.15.72:0/934249508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee54006130 0x7fee540069d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fee50009ac0 tx=0x7fee54007a00 comp rx=0 tx=0).stop 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.192+0000 7fee66d9d640 1 -- 172.21.15.72:0/934249508 reap_dead start 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.192+0000 7fee6cf53640 1 -- 172.21.15.72:0/934249508 shutdown_connections 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.192+0000 7fee6cf53640 1 -- 172.21.15.72:0/934249508 >> 172.21.15.72:0/934249508 conn(0x7fee680b0a90 msgr2=0x7fee680b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.192+0000 7fee6cf53640 1 -- 172.21.15.72:0/934249508 shutdown_connections 2023-12-12T09:54:11.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.192+0000 7fee6cf53640 1 -- 172.21.15.72:0/934249508 wait complete. 2023-12-12T09:54:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.194+0000 7fee6cf53640 1 Processor -- start 2023-12-12T09:54:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.194+0000 7fee6cf53640 1 -- start start 2023-12-12T09:54:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee6cf53640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006130 0x7fee68153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee6cf53640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee680b2830 0x7fee6814a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee6cf53640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee681eeff0 0x7fee681ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee6cf53640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee680b3010 con 0x7fee681eeff0 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee6cf53640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee681540a0 con 0x7fee54006130 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee6cf53640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815d9d0 con 0x7fee680b2830 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee66d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee681eeff0 0x7fee681ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee66d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee681eeff0 0x7fee681ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46650/0 (socket says 172.21.15.72:46650) 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee66d9d640 1 -- 172.21.15.72:0/2033319244 learned_addr learned my addr 172.21.15.72:0/2033319244 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.195+0000 7fee6659c640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006130 0x7fee68153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee5ffff640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee680b2830 0x7fee6814a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee66d9d640 1 -- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee680b2830 msgr2=0x7fee6814a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee66d9d640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee680b2830 0x7fee6814a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee66d9d640 1 -- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006130 msgr2=0x7fee68153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee66d9d640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006130 0x7fee68153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee66d9d640 1 -- 172.21.15.72:0/2033319244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6814ad90 con 0x7fee681eeff0 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee6659c640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006130 0x7fee68153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee66d9d640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee681eeff0 0x7fee681ef470 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fee680b16a0 tx=0x7fee6000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee5effd640 1 -- 172.21.15.72:0/2033319244 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60004d60 con 0x7fee681eeff0 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee5effd640 1 -- 172.21.15.72:0/2033319244 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6000bb50 con 0x7fee681eeff0 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee680b2d30 con 0x7fee681eeff0 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee5effd640 1 -- 172.21.15.72:0/2033319244 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60007070 con 0x7fee681eeff0 2023-12-12T09:54:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.196+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6815be30 con 0x7fee681eeff0 2023-12-12T09:54:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.197+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee680745d0 con 0x7fee681eeff0 2023-12-12T09:54:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.198+0000 7fee5effd640 1 -- 172.21.15.72:0/2033319244 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee600071d0 con 0x7fee681eeff0 2023-12-12T09:54:11.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.199+0000 7fee5effd640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee4806ae40 0x7fee4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.199+0000 7fee5effd640 1 -- 172.21.15.72:0/2033319244 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fee600894c0 con 0x7fee681eeff0 2023-12-12T09:54:11.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.200+0000 7fee6659c640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee4806ae40 0x7fee4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.200+0000 7fee6659c640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee4806ae40 0x7fee4806d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee580016f0 tx=0x7fee58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:11.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.204+0000 7fee5effd640 1 -- 172.21.15.72:0/2033319244 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee60054410 con 0x7fee681eeff0 2023-12-12T09:54:11.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.427+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee680ae290 con 0x7fee681eeff0 2023-12-12T09:54:11.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.427+0000 7fee5effd640 1 -- 172.21.15.72:0/2033319244 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fee60057a30 con 0x7fee681eeff0 2023-12-12T09:54:11.431 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:11.431 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:11.432 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:11.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee4806ae40 msgr2=0x7fee4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee4806ae40 0x7fee4806d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee580016f0 tx=0x7fee58009040 comp rx=0 tx=0).stop 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee681eeff0 msgr2=0x7fee681ef470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee681eeff0 0x7fee681ef470 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fee680b16a0 tx=0x7fee6000b4a0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 shutdown_connections 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee4806ae40 0x7fee4806d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee681eeff0 0x7fee681ef470 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee680b2830 0x7fee6814a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 --2- 172.21.15.72:0/2033319244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee54006130 0x7fee68153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 >> 172.21.15.72:0/2033319244 conn(0x7fee680b0a90 msgr2=0x7fee680cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 shutdown_connections 2023-12-12T09:54:11.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.430+0000 7fee6cf53640 1 -- 172.21.15.72:0/2033319244 wait complete. 2023-12-12T09:54:11.449 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json perf dump purge_queue pq_executed 2023-12-12T09:54:11.691 INFO:teuthology.orchestra.run.smithi113.stdout:{"purge_queue":{"pq_executed":1001}} 2023-12-12T09:54:11.691 DEBUG:tasks.cephfs.filesystem:_json_asok output { "purge_queue": { "pq_executed": 1001 } } 2023-12-12T09:54:11.691 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2023-12-12T09:54:11.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.805+0000 7f7b01b59f40 1 Processor -- start 2023-12-12T09:54:11.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.805+0000 7f7b01b59f40 1 -- start start 2023-12-12T09:54:11.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.806+0000 7f7b01b59f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 0x55ba737275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.806+0000 7f7b01b59f40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 0x55ba73720d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.806+0000 7f7b01b59f40 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba7367e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.806+0000 7f7b01b59f40 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba73603190 con 0x55ba73721280 2023-12-12T09:54:11.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.806+0000 7f7b01b59f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba735fea80 con 0x7f7ae8002300 2023-12-12T09:54:11.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.806+0000 7f7b01b59f40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba735b5bc0 con 0x55ba73729750 2023-12-12T09:54:11.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba7367e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba7367e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46666/0 (socket says 172.21.15.72:46666) 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 -- 172.21.15.72:0/949104865 learned_addr learned my addr 172.21.15.72:0/949104865 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7aff65b640 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 0x55ba737275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 -- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 msgr2=0x55ba73720d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7afee5a640 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 0x55ba73720d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 0x55ba73720d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 -- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 msgr2=0x55ba737275a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 0x55ba737275a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 -- 172.21.15.72:0/949104865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba73597830 con 0x55ba73721280 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7aff65b640 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 0x55ba737275a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7afee5a640 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 0x55ba73720d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.807+0000 7f7affe5c640 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba7367e7b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7af8000c00 tx=0x7f7af800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d38b25055cee5e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7afde58640 1 -- 172.21.15.72:0/949104865 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800f710 con 0x55ba73721280 2023-12-12T09:54:11.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7afde58640 1 -- 172.21.15.72:0/949104865 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af800f870 con 0x55ba73721280 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7afde58640 1 -- 172.21.15.72:0/949104865 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800fc20 con 0x55ba73721280 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 -- 172.21.15.72:0/949104865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 msgr2=0x55ba7367e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba7367e7b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7af8000c00 tx=0x7f7af800e940 comp rx=0 tx=0).stop 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 -- 172.21.15.72:0/949104865 shutdown_connections 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba7367e7b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 0x55ba73720d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 --2- 172.21.15.72:0/949104865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 0x55ba737275a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 -- 172.21.15.72:0/949104865 >> 172.21.15.72:0/949104865 conn(0x55ba7369a720 msgr2=0x55ba73699e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 -- 172.21.15.72:0/949104865 shutdown_connections 2023-12-12T09:54:11.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.808+0000 7f7b01b59f40 1 -- 172.21.15.72:0/949104865 wait complete. 2023-12-12T09:54:11.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.809+0000 7f7b01b59f40 1 Processor -- start 2023-12-12T09:54:11.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.809+0000 7f7b01b59f40 1 -- start start 2023-12-12T09:54:11.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.810+0000 7f7b01b59f40 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba73696390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.810+0000 7f7b01b59f40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 0x55ba73697250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.810+0000 7f7b01b59f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 0x55ba73692c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.810+0000 7f7b01b59f40 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba735989c0 con 0x55ba73721280 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.810+0000 7f7b01b59f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7358c520 con 0x7f7ae8002300 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.810+0000 7f7b01b59f40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba73713fa0 con 0x55ba73729750 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.810+0000 7f7aff65b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba73696390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.810+0000 7f7aff65b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba73696390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46682/0 (socket says 172.21.15.72:46682) 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7aff65b640 1 -- 172.21.15.72:0/3463520563 learned_addr learned my addr 172.21.15.72:0/3463520563 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7afee5a640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 0x55ba73697250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7affe5c640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 0x55ba73692c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7aff65b640 1 -- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 msgr2=0x55ba73697250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7aff65b640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba73729750 0x55ba73697250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7aff65b640 1 -- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 msgr2=0x55ba73692c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7aff65b640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ae8002300 0x55ba73692c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7aff65b640 1 -- 172.21.15.72:0/3463520563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba735a4680 con 0x55ba73721280 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.811+0000 7f7aff65b640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba73696390 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7af0000f80 tx=0x7f7ae80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.812+0000 7f7afce56640 1 -- 172.21.15.72:0/3463520563 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af004aaf0 con 0x55ba73721280 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.812+0000 7f7afce56640 1 -- 172.21.15.72:0/3463520563 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af005e480 con 0x55ba73721280 2023-12-12T09:54:11.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.812+0000 7f7afce56640 1 -- 172.21.15.72:0/3463520563 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af005e810 con 0x55ba73721280 2023-12-12T09:54:11.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.812+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ba73597290 con 0x55ba73721280 2023-12-12T09:54:11.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.812+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba7369ae60 con 0x55ba73721280 2023-12-12T09:54:11.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.814+0000 7f7afce56640 1 -- 172.21.15.72:0/3463520563 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7af005e970 con 0x55ba73721280 2023-12-12T09:54:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.815+0000 7f7afce56640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7aec069850 0x7f7aec06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.815+0000 7f7afce56640 1 -- 172.21.15.72:0/3463520563 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7af00c9f50 con 0x55ba73721280 2023-12-12T09:54:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.815+0000 7f7afee5a640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7aec069850 0x7f7aec06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.815+0000 7f7b01b59f40 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x55ba73697790 0x55ba737c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.815+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 --> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ba736968d0 con 0x55ba73697790 2023-12-12T09:54:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.816+0000 7f7afee5a640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7aec069850 0x7f7aec06bd10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005400 tx=0x7f7af4005a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.816+0000 7f7affe5c640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x55ba73697790 0x55ba737c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:11.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.817+0000 7f7affe5c640 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x55ba73697790 0x55ba737c1bb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7affe5c640 1 -- 172.21.15.72:0/3463520563 <== osd.7 v2:172.21.15.113:6824/1904806807 1 ==== osd_op_reply(1 10000000000.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7af8014030 con 0x55ba73697790 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr: error stat-ing cephfs_metadata/10000000000.00000000: (2) No such file or directory 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x55ba73697790 msgr2=0x55ba737c1bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x55ba73697790 0x55ba737c1bb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7aec069850 msgr2=0x7f7aec06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7aec069850 0x7f7aec06bd10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005400 tx=0x7f7af4005a60 comp rx=0 tx=0).stop 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 msgr2=0x55ba73696390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 --2- 172.21.15.72:0/3463520563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ba73721280 0x55ba73696390 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7af0000f80 tx=0x7f7ae80040e0 comp rx=0 tx=0).stop 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7affe5c640 1 -- 172.21.15.72:0/3463520563 reap_dead start 2023-12-12T09:54:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 shutdown_connections 2023-12-12T09:54:11.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 >> 172.21.15.72:0/3463520563 conn(0x55ba7369a720 msgr2=0x55ba7367e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:11.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 shutdown_connections 2023-12-12T09:54:11.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:11.818+0000 7f7b01b59f40 1 -- 172.21.15.72:0/3463520563 wait complete. 2023-12-12T09:54:11.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:54:11.824 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2023-12-12T09:54:12.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.015+0000 7fcf4f2d7640 1 Processor -- start 2023-12-12T09:54:12.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.016+0000 7fcf4f2d7640 1 -- start start 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.017+0000 7fcf4f2d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 0x7fcf480b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.017+0000 7fcf4f2d7640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fcf480cfed0 0x7fcf480ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.017+0000 7fcf4f2d7640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf480d20e0 con 0x7fcf480b4190 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.017+0000 7fcf4f2d7640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf480d3d80 con 0x7fcf480cfed0 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.017+0000 7fcf4f2d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480d1840 con 0x7fcf4815d560 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.017+0000 7fcf4dad4640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fcf480b4190 0x7fcf480cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:44940/0 (socket says 172.21.15.72:44940) 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.017+0000 7fcf4dad4640 1 -- 172.21.15.72:0/1276896006 learned_addr learned my addr 172.21.15.72:0/1276896006 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.017+0000 7fcf4ead6640 1 --2- 172.21.15.72:0/1276896006 >> v2:172.21.15.72:3300/0 conn(0x7fcf480cfed0 0x7fcf480ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.018+0000 7fcf4e2d5640 1 --2- 172.21.15.72:0/1276896006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 0x7fcf480b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.018+0000 7fcf4d2d3640 1 -- 172.21.15.72:0/1276896006 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038288029 0 0) 0x7fcf480d20e0 con 0x7fcf480b4190 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.018+0000 7fcf4d2d3640 1 -- 172.21.15.72:0/1276896006 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf34003520 con 0x7fcf480b4190 2023-12-12T09:54:12.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.018+0000 7fcf4ead6640 1 -- 172.21.15.72:0/1276896006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 msgr2=0x7fcf480b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:12.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.018+0000 7fcf4ead6640 1 --2- 172.21.15.72:0/1276896006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 0x7fcf480b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.018+0000 7fcf4ead6640 1 -- 172.21.15.72:0/1276896006 >> v1:172.21.15.72:6789/0 conn(0x7fcf480b4190 legacy=0x7fcf480cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.018+0000 7fcf4ead6640 1 -- 172.21.15.72:0/1276896006 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf480d1b20 con 0x7fcf480cfed0 2023-12-12T09:54:12.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.019+0000 7fcf4ead6640 1 --2- 172.21.15.72:0/1276896006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf480ce150 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009d90 tx=0x7fcf4002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c331d274eb40faf server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:12.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.019+0000 7fcf4d2d3640 1 -- 172.21.15.72:0/1276896006 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4002cc50 con 0x7fcf480cfed0 2023-12-12T09:54:12.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.019+0000 7fcf4d2d3640 1 -- 172.21.15.72:0/1276896006 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4002cdb0 con 0x7fcf480cfed0 2023-12-12T09:54:12.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.019+0000 7fcf4d2d3640 1 -- 172.21.15.72:0/1276896006 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40034660 con 0x7fcf480cfed0 2023-12-12T09:54:12.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.020+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1276896006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 msgr2=0x7fcf480ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.020+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1276896006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf480ce150 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009d90 tx=0x7fcf4002bea0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.020+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1276896006 shutdown_connections 2023-12-12T09:54:12.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.020+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1276896006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf480ce150 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.020+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1276896006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 0x7fcf480b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.020+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1276896006 >> 172.21.15.72:0/1276896006 conn(0x7fcf4809c220 msgr2=0x7fcf480d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:12.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.020+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1276896006 shutdown_connections 2023-12-12T09:54:12.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.020+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1276896006 wait complete. 2023-12-12T09:54:12.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.022+0000 7fcf4f2d7640 1 Processor -- start 2023-12-12T09:54:12.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.022+0000 7fcf4f2d7640 1 -- start start 2023-12-12T09:54:12.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4f2d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf480b4190 0x7fcf481580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4f2d7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf48159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4f2d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 0x7fcf4815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4f2d7640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf48158620 con 0x7fcf480cfed0 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4f2d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf481f47c0 con 0x7fcf480b4190 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4f2d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480b0300 con 0x7fcf4815d560 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4dad4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf48159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4dad4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf48159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46702/0 (socket says 172.21.15.72:46702) 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.023+0000 7fcf4dad4640 1 -- 172.21.15.72:0/1099448149 learned_addr learned my addr 172.21.15.72:0/1099448149 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf4e2d5640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf480b4190 0x7fcf481580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf4dad4640 1 -- 172.21.15.72:0/1099448149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 msgr2=0x7fcf4815a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf4dad4640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 0x7fcf4815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf4dad4640 1 -- 172.21.15.72:0/1099448149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf480b4190 msgr2=0x7fcf481580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf4dad4640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf480b4190 0x7fcf481580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf4dad4640 1 -- 172.21.15.72:0/1099448149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40009a70 con 0x7fcf480cfed0 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf4dad4640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf48159ef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00c500 tx=0x7fcf3c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf3affd640 1 -- 172.21.15.72:0/1099448149 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c005ce0 con 0x7fcf480cfed0 2023-12-12T09:54:12.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf3affd640 1 -- 172.21.15.72:0/1099448149 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c005e40 con 0x7fcf480cfed0 2023-12-12T09:54:12.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf3affd640 1 -- 172.21.15.72:0/1099448149 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c007330 con 0x7fcf480cfed0 2023-12-12T09:54:12.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.024+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf480b05e0 con 0x7fcf480cfed0 2023-12-12T09:54:12.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.025+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf480b0ba0 con 0x7fcf480cfed0 2023-12-12T09:54:12.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.026+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf48074450 con 0x7fcf480cfed0 2023-12-12T09:54:12.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.026+0000 7fcf3affd640 1 -- 172.21.15.72:0/1099448149 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf3c007490 con 0x7fcf480cfed0 2023-12-12T09:54:12.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.027+0000 7fcf3affd640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcf2c06ae40 0x7fcf2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.027+0000 7fcf3affd640 1 -- 172.21.15.72:0/1099448149 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcf3c088e10 con 0x7fcf480cfed0 2023-12-12T09:54:12.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.028+0000 7fcf4e2d5640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcf2c06ae40 0x7fcf2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.028+0000 7fcf4e2d5640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcf2c06ae40 0x7fcf2c06d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf44006fd0 tx=0x7fcf44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:12.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.032+0000 7fcf3affd640 1 -- 172.21.15.72:0/1099448149 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf3c08c030 con 0x7fcf480cfed0 2023-12-12T09:54:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.290+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fcf480c6430 con 0x7fcf480cfed0 2023-12-12T09:54:12.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.291+0000 7fcf3affd640 1 -- 172.21.15.72:0/1099448149 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1657 (secure 0 0 0) 0x7fcf3c017020 con 0x7fcf480cfed0 2023-12-12T09:54:12.292 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "stats": { 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "total_bytes": 773094113280, 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "total_avail_bytes": 772840312832, 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "total_used_bytes": 253800448, 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "total_used_raw_bytes": 253800448, 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "total_used_raw_ratio": 0.00032829178962856531, 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "num_osds": 8, 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "num_per_pool_osds": 8, 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "num_per_pool_omap_osds": 8 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "stats_by_class": { 2023-12-12T09:54:12.293 INFO:teuthology.orchestra.run.smithi072.stdout: "ssd": { 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "total_bytes": 773094113280, 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "total_avail_bytes": 772840312832, 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "total_used_bytes": 253800448, 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "total_used_raw_bytes": 253800448, 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "total_used_raw_ratio": 0.00032829178962856531 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "pools": [ 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "name": ".mgr", 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 1, 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "stats": { 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "stored": 590368, 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": 2, 2023-12-12T09:54:12.294 INFO:teuthology.orchestra.run.smithi072.stdout: "kb_used": 1160, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "bytes_used": 1187840, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "percent_used": 1.6179368458324461e-06, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "max_avail": 367084208128 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephfs_metadata", 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 5, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "stats": { 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "stored": 995375, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": 23, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "kb_used": 2000, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "bytes_used": 2048000, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "percent_used": 2.7895430321223103e-06, 2023-12-12T09:54:12.295 INFO:teuthology.orchestra.run.smithi072.stdout: "max_avail": 367084208128 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephfs_data", 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 6, 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "stats": { 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "stored": 0, 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": 0, 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "kb_used": 0, 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "bytes_used": 0, 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "percent_used": 0, 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: "max_avail": 367084208128 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:12.296 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2023-12-12T09:54:12.297 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcf2c06ae40 msgr2=0x7fcf2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcf2c06ae40 0x7fcf2c06d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf44006fd0 tx=0x7fcf44008040 comp rx=0 tx=0).stop 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 msgr2=0x7fcf48159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf48159ef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00c500 tx=0x7fcf3c00c9c0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 shutdown_connections 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcf2c06ae40 0x7fcf2c06d300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcf4815d560 0x7fcf4815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf480cfed0 0x7fcf48159ef0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 --2- 172.21.15.72:0/1099448149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf480b4190 0x7fcf481580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 >> 172.21.15.72:0/1099448149 conn(0x7fcf4809c220 msgr2=0x7fcf480ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 shutdown_connections 2023-12-12T09:54:12.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.293+0000 7fcf4f2d7640 1 -- 172.21.15.72:0/1099448149 wait complete. 2023-12-12T09:54:12.313 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-12T09:54:12.313 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-12T09:54:12.314 DEBUG:teuthology.orchestra.run.smithi072:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-12T09:54:12.423 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-12T09:54:12.539 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-12T09:54:12.540 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:12.540 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:12.601 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:12.601 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:12.655 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:12.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.869+0000 7fc0a6143640 1 Processor -- start 2023-12-12T09:54:12.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.870+0000 7fc0a6143640 1 -- start start 2023-12-12T09:54:12.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.871+0000 7fc0a6143640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0a0159fd0 0x7fc0a00ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.871+0000 7fc0a6143640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0070750 0x7fc0a0070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.871+0000 7fc0a6143640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a0070070 con 0x7fc0a00711e0 2023-12-12T09:54:12.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.871+0000 7fc0a6143640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a00705f0 con 0x7fc0a0159fd0 2023-12-12T09:54:12.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.871+0000 7fc0a6143640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0071650 con 0x7fc0a0070750 2023-12-12T09:54:12.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.872+0000 7fc0a5141640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0a0159fd0 0x7fc0a00ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.872+0000 7fc0a5141640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0a0159fd0 0x7fc0a00ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:37152/0 (socket says 172.21.15.72:37152) 2023-12-12T09:54:12.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.872+0000 7fc0a5141640 1 -- 172.21.15.72:0/400899584 learned_addr learned my addr 172.21.15.72:0/400899584 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:12.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.872+0000 7fc0a4940640 1 --2- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0070750 0x7fc0a0070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.873+0000 7fc0a5141640 1 -- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0070750 msgr2=0x7fc0a0070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:12.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.873+0000 7fc0a5141640 1 --2- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0070750 0x7fc0a0070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.873+0000 7fc0a5141640 1 -- 172.21.15.72:0/400899584 >> v1:172.21.15.113:6789/0 conn(0x7fc0a00711e0 legacy=0x7fc0a00ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.873+0000 7fc0a5141640 1 -- 172.21.15.72:0/400899584 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a0070310 con 0x7fc0a0159fd0 2023-12-12T09:54:12.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.873+0000 7fc0a4940640 1 --2- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0070750 0x7fc0a0070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:54:12.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.874+0000 7fc0a5141640 1 --2- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0159fd0 0x7fc0a00ae1d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009870 tx=0x7fc09c02ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21637fc51125637e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:12.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.874+0000 7fc097fff640 1 -- 172.21.15.72:0/400899584 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c02f840 con 0x7fc0a0159fd0 2023-12-12T09:54:12.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc097fff640 1 -- 172.21.15.72:0/400899584 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c02f9a0 con 0x7fc0a0159fd0 2023-12-12T09:54:12.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc097fff640 1 -- 172.21.15.72:0/400899584 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c02fd30 con 0x7fc0a0159fd0 2023-12-12T09:54:12.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc0a6143640 1 -- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0159fd0 msgr2=0x7fc0a00ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc0a6143640 1 --2- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0159fd0 0x7fc0a00ae1d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009870 tx=0x7fc09c02ba20 comp rx=0 tx=0).stop 2023-12-12T09:54:12.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc0a6143640 1 -- 172.21.15.72:0/400899584 shutdown_connections 2023-12-12T09:54:12.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc0a6143640 1 --2- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0070750 0x7fc0a0070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc0a6143640 1 --2- 172.21.15.72:0/400899584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0159fd0 0x7fc0a00ae1d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc0a6143640 1 -- 172.21.15.72:0/400899584 >> 172.21.15.72:0/400899584 conn(0x7fc0a00b0860 msgr2=0x7fc0a00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc0a6143640 1 -- 172.21.15.72:0/400899584 shutdown_connections 2023-12-12T09:54:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.875+0000 7fc0a6143640 1 -- 172.21.15.72:0/400899584 wait complete. 2023-12-12T09:54:12.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.877+0000 7fc0a6143640 1 Processor -- start 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.877+0000 7fc0a6143640 1 -- start start 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.878+0000 7fc0a6143640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0070750 0x7fc0a01553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.878+0000 7fc0a6143640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0a00711e0 0x7fc0a0148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.878+0000 7fc0a6143640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0159fd0 0x7fc0a0148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.878+0000 7fc0a6143640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01491c0 con 0x7fc0a00711e0 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.878+0000 7fc0a6143640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01494a0 con 0x7fc0a0070750 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.878+0000 7fc0a6143640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0149780 con 0x7fc0a0159fd0 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a4940640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0a00711e0 0x7fc0a0148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a4940640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0a00711e0 0x7fc0a0148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46708/0 (socket says 172.21.15.72:46708) 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a4940640 1 -- 172.21.15.72:0/3394247096 learned_addr learned my addr 172.21.15.72:0/3394247096 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a5141640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0070750 0x7fc0a01553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a5942640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0159fd0 0x7fc0a0148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a4940640 1 -- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0159fd0 msgr2=0x7fc0a0148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a4940640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0159fd0 0x7fc0a0148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a4940640 1 -- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0070750 msgr2=0x7fc0a01553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a4940640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0070750 0x7fc0a01553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a4940640 1 -- 172.21.15.72:0/3394247096 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09000ba70 con 0x7fc0a00711e0 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a5942640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0159fd0 0x7fc0a0148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.879+0000 7fc0a5141640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0070750 0x7fc0a01553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.880+0000 7fc0a4940640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0a00711e0 0x7fc0a0148570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc0900089e0 tx=0x7fc090008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.880+0000 7fc095ffb640 1 -- 172.21.15.72:0/3394247096 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090010040 con 0x7fc0a00711e0 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.880+0000 7fc095ffb640 1 -- 172.21.15.72:0/3394247096 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090009d20 con 0x7fc0a00711e0 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.880+0000 7fc095ffb640 1 -- 172.21.15.72:0/3394247096 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090004270 con 0x7fc0a00711e0 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.880+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c0094d0 con 0x7fc0a00711e0 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.880+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a0149d40 con 0x7fc0a00711e0 2023-12-12T09:54:12.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.881+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a0074450 con 0x7fc0a00711e0 2023-12-12T09:54:12.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.882+0000 7fc095ffb640 1 -- 172.21.15.72:0/3394247096 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0900043d0 con 0x7fc0a00711e0 2023-12-12T09:54:12.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.883+0000 7fc095ffb640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc08406ae40 0x7fc08406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:12.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.883+0000 7fc095ffb640 1 -- 172.21.15.72:0/3394247096 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc0900895d0 con 0x7fc0a00711e0 2023-12-12T09:54:12.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.883+0000 7fc0a5141640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc08406ae40 0x7fc08406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:12.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.884+0000 7fc0a5141640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc08406ae40 0x7fc08406d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc09c036000 tx=0x7fc09c039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:12.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:12.888+0000 7fc095ffb640 1 -- 172.21.15.72:0/3394247096 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc090054530 con 0x7fc0a00711e0 2023-12-12T09:54:13.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.079+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0a006c310 con 0x7fc0a00711e0 2023-12-12T09:54:13.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.082+0000 7fc095ffb640 1 -- 172.21.15.72:0/3394247096 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16823 (secure 0 0 0) 0x7fc090057b50 con 0x7fc0a00711e0 2023-12-12T09:54:13.087 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:13.087 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":40,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:53:56.442722+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:53:45.083566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-12T09:53:46.274070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:13.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc08406ae40 msgr2=0x7fc08406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc08406ae40 0x7fc08406d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc09c036000 tx=0x7fc09c039040 comp rx=0 tx=0).stop 2023-12-12T09:54:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0a00711e0 msgr2=0x7fc0a0148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0a00711e0 0x7fc0a0148570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc0900089e0 tx=0x7fc090008ad0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 shutdown_connections 2023-12-12T09:54:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc08406ae40 0x7fc08406d300 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0a0159fd0 0x7fc0a0148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0a00711e0 0x7fc0a0148570 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 --2- 172.21.15.72:0/3394247096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0a0070750 0x7fc0a01553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 >> 172.21.15.72:0/3394247096 conn(0x7fc0a00b0860 msgr2=0x7fc0a00aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:13.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 shutdown_connections 2023-12-12T09:54:13.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.085+0000 7fc0a6143640 1 -- 172.21.15.72:0/3394247096 wait complete. 2023-12-12T09:54:13.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:53:45.083566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:53:46.274070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:54:13.109 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:54:13.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.316+0000 7f1e3a635640 1 Processor -- start 2023-12-12T09:54:13.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.316+0000 7f1e3a635640 1 -- start start 2023-12-12T09:54:13.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.318+0000 7f1e3a635640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e340ce150 0x7f1e340ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.318+0000 7f1e3a635640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e340cedf0 0x7f1e340cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.318+0000 7f1e3a635640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3406c500 con 0x7f1e34155400 2023-12-12T09:54:13.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.318+0000 7f1e3a635640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e340cf7c0 con 0x7f1e340cedf0 2023-12-12T09:54:13.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.318+0000 7f1e3a635640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3415b4a0 con 0x7f1e340ce150 2023-12-12T09:54:13.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.318+0000 7f1e39633640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1e34155400 0x7f1e3406bdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:44956/0 (socket says 172.21.15.72:44956) 2023-12-12T09:54:13.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.318+0000 7f1e39633640 1 -- 172.21.15.72:0/1168937068 learned_addr learned my addr 172.21.15.72:0/1168937068 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:13.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.318+0000 7f1e39e34640 1 --2- 172.21.15.72:0/1168937068 >> v2:172.21.15.113:3300/0 conn(0x7f1e340cedf0 0x7f1e340cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299191831 0 0) 0x7f1e3406c500 con 0x7f1e34155400 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e38e32640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e340ce150 0x7f1e340ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e20003520 con 0x7f1e34155400 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335171340 0 0) 0x7f1e20003520 con 0x7f1e34155400 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e340ce150 msgr2=0x7f1e340ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e340ce150 0x7f1e340ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 >> v2:172.21.15.113:3300/0 conn(0x7f1e340cedf0 msgr2=0x7f1e340cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 --2- 172.21.15.72:0/1168937068 >> v2:172.21.15.113:3300/0 conn(0x7f1e340cedf0 0x7f1e340cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e340adbb0 con 0x7f1e34155400 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f1e30001670 con 0x7f1e34155400 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 >> v1:172.21.15.72:6789/0 conn(0x7f1e34155400 legacy=0x7f1e3406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e20003970 0x7f1e20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e340ce150 0x7f1e200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20004d00 0x7f1e20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340cf7c0 con 0x7f1e20003970 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3415b4a0 con 0x7f1e20004d00 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2000b980 con 0x7f1e340ce150 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.319+0000 7f1e39633640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e20003970 0x7f1e20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.320+0000 7f1e38e32640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e340ce150 0x7f1e200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.320+0000 7f1e39e34640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20004d00 0x7f1e20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.320+0000 7f1e39633640 1 -- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e340ce150 msgr2=0x7f1e200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.320+0000 7f1e39633640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e340ce150 0x7f1e200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.320+0000 7f1e39633640 1 -- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20004d00 msgr2=0x7f1e20005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.320+0000 7f1e39633640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20004d00 0x7f1e20005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.320+0000 7f1e39633640 1 -- 172.21.15.72:0/1168937068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e20003520 con 0x7f1e20003970 2023-12-12T09:54:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.320+0000 7f1e39e34640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20004d00 0x7f1e20005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:13.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.321+0000 7f1e39633640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e20003970 0x7f1e20003d90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c110 tx=0x7f1e200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce99f34800018f39 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.321+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30005ad0 con 0x7f1e20003970 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.321+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30005c50 con 0x7f1e20003970 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.321+0000 7f1e2bfff640 1 -- 172.21.15.72:0/1168937068 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3002c830 con 0x7f1e20003970 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.322+0000 7f1e3a635640 1 -- 172.21.15.72:0/1168937068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e20003970 msgr2=0x7f1e20003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.322+0000 7f1e3a635640 1 --2- 172.21.15.72:0/1168937068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e20003970 0x7f1e20003d90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c110 tx=0x7f1e200057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.322+0000 7f1e39e34640 1 -- 172.21.15.72:0/1168937068 reap_dead start 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.322+0000 7f1e3a635640 1 -- 172.21.15.72:0/1168937068 shutdown_connections 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.322+0000 7f1e3a635640 1 -- 172.21.15.72:0/1168937068 >> 172.21.15.72:0/1168937068 conn(0x7f1e3406a640 msgr2=0x7f1e3406aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.322+0000 7f1e3a635640 1 -- 172.21.15.72:0/1168937068 shutdown_connections 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.322+0000 7f1e3a635640 1 -- 172.21.15.72:0/1168937068 wait complete. 2023-12-12T09:54:13.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.324+0000 7f1e3a635640 1 Processor -- start 2023-12-12T09:54:13.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.324+0000 7f1e3a635640 1 -- start start 2023-12-12T09:54:13.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e3a635640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20003970 0x7f1e3409c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e3a635640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e3414a4c0 0x7f1e3409a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e3a635640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3409ab00 0x7f1e340b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e3a635640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3409b500 con 0x7f1e3414a4c0 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e3a635640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3409b7e0 con 0x7f1e20003970 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e3a635640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3409bac0 con 0x7f1e3409ab00 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e38e32640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e3414a4c0 0x7f1e3409a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e38e32640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e3414a4c0 0x7f1e3409a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46714/0 (socket says 172.21.15.72:46714) 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.325+0000 7f1e38e32640 1 -- 172.21.15.72:0/2916767124 learned_addr learned my addr 172.21.15.72:0/2916767124 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e39633640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20003970 0x7f1e3409c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e39e34640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3409ab00 0x7f1e340b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e38e32640 1 -- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3409ab00 msgr2=0x7f1e340b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e38e32640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3409ab00 0x7f1e340b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e38e32640 1 -- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20003970 msgr2=0x7f1e3409c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e38e32640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20003970 0x7f1e3409c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e38e32640 1 -- 172.21.15.72:0/2916767124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3409bda0 con 0x7f1e3414a4c0 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e39e34640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3409ab00 0x7f1e340b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e39633640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20003970 0x7f1e3409c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e38e32640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e3414a4c0 0x7f1e3409a5c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1e24011600 tx=0x7f1e24011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e29ffb640 1 -- 172.21.15.72:0/2916767124 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400f920 con 0x7f1e3414a4c0 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e29ffb640 1 -- 172.21.15.72:0/2916767124 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2400fa80 con 0x7f1e3414a4c0 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e29ffb640 1 -- 172.21.15.72:0/2916767124 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400fdf0 con 0x7f1e3414a4c0 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.326+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3409c080 con 0x7f1e3414a4c0 2023-12-12T09:54:13.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.327+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3414a940 con 0x7f1e3414a4c0 2023-12-12T09:54:13.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.328+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e34074450 con 0x7f1e3414a4c0 2023-12-12T09:54:13.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.329+0000 7f1e29ffb640 1 -- 172.21.15.72:0/2916767124 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e24015460 con 0x7f1e3414a4c0 2023-12-12T09:54:13.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.329+0000 7f1e29ffb640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1e1806ae40 0x7f1e1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.330+0000 7f1e29ffb640 1 -- 172.21.15.72:0/2916767124 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1e24089cf0 con 0x7f1e3414a4c0 2023-12-12T09:54:13.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.330+0000 7f1e39633640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1e1806ae40 0x7f1e1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.330+0000 7f1e39633640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1e1806ae40 0x7f1e1806d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1e30034000 tx=0x7f1e30007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:13.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.334+0000 7f1e29ffb640 1 -- 172.21.15.72:0/2916767124 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e24054ba0 con 0x7f1e3414a4c0 2023-12-12T09:54:13.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.528+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e3406d750 con 0x7f1e3414a4c0 2023-12-12T09:54:13.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.529+0000 7f1e29ffb640 1 -- 172.21.15.72:0/2916767124 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e240220a0 con 0x7f1e3414a4c0 2023-12-12T09:54:13.531 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:54:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.531+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1e1806ae40 msgr2=0x7f1e1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.531+0000 7f1e3a635640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1e1806ae40 0x7f1e1806d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1e30034000 tx=0x7f1e30007000 comp rx=0 tx=0).stop 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.531+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e3414a4c0 msgr2=0x7f1e3409a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.531+0000 7f1e3a635640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e3414a4c0 0x7f1e3409a5c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1e24011600 tx=0x7f1e24011ac0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.532+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 shutdown_connections 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.532+0000 7f1e3a635640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1e1806ae40 0x7f1e1806d300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.532+0000 7f1e3a635640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3409ab00 0x7f1e340b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.532+0000 7f1e3a635640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e3414a4c0 0x7f1e3409a5c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.532+0000 7f1e3a635640 1 --2- 172.21.15.72:0/2916767124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e20003970 0x7f1e3409c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.532+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 >> 172.21.15.72:0/2916767124 conn(0x7f1e3406a640 msgr2=0x7f1e3415d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.532+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 shutdown_connections 2023-12-12T09:54:13.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.532+0000 7f1e3a635640 1 -- 172.21.15.72:0/2916767124 wait complete. 2023-12-12T09:54:13.552 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:54:13.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.774+0000 7f9c90bb0640 1 Processor -- start 2023-12-12T09:54:13.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.775+0000 7f9c90bb0640 1 -- start start 2023-12-12T09:54:13.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.776+0000 7f9c90bb0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.776+0000 7f9c90bb0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c8c0b00a0 0x7f9c8c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.776+0000 7f9c90bb0640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8c0b2390 con 0x7f9c8c0d14e0 2023-12-12T09:54:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.776+0000 7f9c90bb0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c0ce560 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.776+0000 7f9c90bb0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0ce830 con 0x7f9c8c0cdbc0 2023-12-12T09:54:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.776+0000 7f9c8a59c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f9c8c0d14e0 0x7f9c8c0d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:44970/0 (socket says 172.21.15.72:44970) 2023-12-12T09:54:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.776+0000 7f9c8a59c640 1 -- 172.21.15.72:0/1842286571 learned_addr learned my addr 172.21.15.72:0/1842286571 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.777+0000 7f9c8ad9d640 1 --2- 172.21.15.72:0/1842286571 >> v2:172.21.15.113:3300/0 conn(0x7f9c8c0b00a0 0x7f9c8c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.777+0000 7f9c83fff640 1 --2- 172.21.15.72:0/1842286571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.778+0000 7f9c83fff640 1 -- 172.21.15.72:0/1842286571 >> v2:172.21.15.113:3300/0 conn(0x7f9c8c0b00a0 msgr2=0x7f9c8c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.778+0000 7f9c83fff640 1 --2- 172.21.15.72:0/1842286571 >> v2:172.21.15.113:3300/0 conn(0x7f9c8c0b00a0 0x7f9c8c0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.778+0000 7f9c83fff640 1 -- 172.21.15.72:0/1842286571 >> v1:172.21.15.72:6789/0 conn(0x7f9c8c0d14e0 legacy=0x7f9c8c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.778+0000 7f9c83fff640 1 -- 172.21.15.72:0/1842286571 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c0ceb10 con 0x7f9c8c0cdbc0 2023-12-12T09:54:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.778+0000 7f9c89d9b640 1 -- 172.21.15.72:0/1842286571 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251556670 0 0) 0x7f9c8c0b2390 con 0x7f9c8c0d14e0 2023-12-12T09:54:13.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.779+0000 7f9c83fff640 1 --2- 172.21.15.72:0/1842286571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0ce020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c740097a0 tx=0x7f9c7402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=837ab5cd4c82c6aa server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:13.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.780+0000 7f9c89d9b640 1 -- 172.21.15.72:0/1842286571 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7402d040 con 0x7f9c8c0cdbc0 2023-12-12T09:54:13.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.780+0000 7f9c89d9b640 1 -- 172.21.15.72:0/1842286571 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c74031810 con 0x7f9c8c0cdbc0 2023-12-12T09:54:13.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.780+0000 7f9c89d9b640 1 -- 172.21.15.72:0/1842286571 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74031ba0 con 0x7f9c8c0cdbc0 2023-12-12T09:54:13.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.781+0000 7f9c90bb0640 1 -- 172.21.15.72:0/1842286571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0cdbc0 msgr2=0x7f9c8c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.781+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/1842286571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0ce020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c740097a0 tx=0x7f9c7402b8b0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.781+0000 7f9c90bb0640 1 -- 172.21.15.72:0/1842286571 shutdown_connections 2023-12-12T09:54:13.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.781+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/1842286571 >> v2:172.21.15.113:3300/0 conn(0x7f9c8c0b00a0 0x7f9c8c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.781+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/1842286571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0ce020 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.781+0000 7f9c90bb0640 1 -- 172.21.15.72:0/1842286571 >> 172.21.15.72:0/1842286571 conn(0x7f9c8c09c3f0 msgr2=0x7f9c8c0d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:13.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.781+0000 7f9c90bb0640 1 -- 172.21.15.72:0/1842286571 shutdown_connections 2023-12-12T09:54:13.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.781+0000 7f9c90bb0640 1 -- 172.21.15.72:0/1842286571 wait complete. 2023-12-12T09:54:13.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.783+0000 7f9c90bb0640 1 Processor -- start 2023-12-12T09:54:13.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.783+0000 7f9c90bb0640 1 -- start start 2023-12-12T09:54:13.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.784+0000 7f9c90bb0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c8c0b00a0 0x7f9c8c151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.784+0000 7f9c90bb0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.784+0000 7f9c90bb0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0d14e0 0x7f9c8c0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.784+0000 7f9c90bb0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c1f6af0 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.784+0000 7f9c90bb0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0b2700 con 0x7f9c8c0cdbc0 2023-12-12T09:54:13.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.784+0000 7f9c90bb0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c0b29e0 con 0x7f9c8c0d14e0 2023-12-12T09:54:13.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.784+0000 7f9c8a59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c8c0b00a0 0x7f9c8c151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8a59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c8c0b00a0 0x7f9c8c151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46720/0 (socket says 172.21.15.72:46720) 2023-12-12T09:54:13.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8a59c640 1 -- 172.21.15.72:0/3794175630 learned_addr learned my addr 172.21.15.72:0/3794175630 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:13.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8ad9d640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0d14e0 0x7f9c8c0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c83fff640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8a59c640 1 -- 172.21.15.72:0/3794175630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0d14e0 msgr2=0x7f9c8c0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8a59c640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0d14e0 0x7f9c8c0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8a59c640 1 -- 172.21.15.72:0/3794175630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8c0cdbc0 msgr2=0x7f9c8c0c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8a59c640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8a59c640 1 -- 172.21.15.72:0/3794175630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c74009400 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.785+0000 7f9c8a59c640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c8c0b00a0 0x7f9c8c151480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c430 tx=0x7f9c7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.786+0000 7f9c82ffd640 1 -- 172.21.15.72:0/3794175630 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0074e0 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.786+0000 7f9c82ffd640 1 -- 172.21.15.72:0/3794175630 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c007660 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.786+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c0b2cc0 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.786+0000 7f9c82ffd640 1 -- 172.21.15.72:0/3794175630 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c004a20 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.786+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c0c8420 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.787+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c0b2e50 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.788+0000 7f9c82ffd640 1 -- 172.21.15.72:0/3794175630 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c7c004b80 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.789+0000 7f9c82ffd640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9c6c06ae40 0x7f9c6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:13.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.789+0000 7f9c82ffd640 1 -- 172.21.15.72:0/3794175630 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9c7c089600 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.789+0000 7f9c83fff640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9c6c06ae40 0x7f9c6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:13.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.790+0000 7f9c83fff640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9c6c06ae40 0x7f9c6c06d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c74036000 tx=0x7f9c7403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:13.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.794+0000 7f9c82ffd640 1 -- 172.21.15.72:0/3794175630 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c7c054550 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.983+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c8c074450 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.984+0000 7f9c82ffd640 1 -- 172.21.15.72:0/3794175630 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c7c057b70 con 0x7f9c8c0b00a0 2023-12-12T09:54:13.987 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.986+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9c6c06ae40 msgr2=0x7f9c6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.986+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9c6c06ae40 0x7f9c6c06d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c74036000 tx=0x7f9c7403c040 comp rx=0 tx=0).stop 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.986+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c8c0b00a0 msgr2=0x7f9c8c151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.986+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c8c0b00a0 0x7f9c8c151480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c430 tx=0x7f9c7c00c8f0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.987+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 shutdown_connections 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.987+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9c6c06ae40 0x7f9c6c06d300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.987+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8c0d14e0 0x7f9c8c0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.987+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8c0cdbc0 0x7f9c8c0c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.987+0000 7f9c90bb0640 1 --2- 172.21.15.72:0/3794175630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c8c0b00a0 0x7f9c8c151480 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:13.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.987+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 >> 172.21.15.72:0/3794175630 conn(0x7f9c8c09c3f0 msgr2=0x7f9c8c0d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:13.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.987+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 shutdown_connections 2023-12-12T09:54:13.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:13.987+0000 7f9c90bb0640 1 -- 172.21.15.72:0/3794175630 wait complete. 2023-12-12T09:54:14.009 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:54:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.216+0000 7f7d9483d640 1 Processor -- start 2023-12-12T09:54:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.216+0000 7f7d9483d640 1 -- start start 2023-12-12T09:54:14.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d9483d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 0x7f7d9006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d9483d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7d9006a460 0x7f7d9006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d9483d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d90071cd0 con 0x7f7d900ce7e0 2023-12-12T09:54:14.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d9483d640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9006ae30 con 0x7f7d9006a460 2023-12-12T09:54:14.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d9483d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900717e0 con 0x7f7d9015af50 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d8ed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7d9006a460 0x7f7d9006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d8dd9b640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f7d900ce7e0 0x7f7d9015ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:44974/0 (socket says 172.21.15.72:44974) 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d8ed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7d9006a460 0x7f7d9006a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46724/0 (socket says 172.21.15.72:46724) 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d8dd9b640 1 -- 172.21.15.72:0/1495429560 learned_addr learned my addr 172.21.15.72:0/1495429560 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.218+0000 7f7d8e59c640 1 --2- 172.21.15.72:0/1495429560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 0x7f7d9006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.219+0000 7f7d8ed9d640 1 -- 172.21.15.72:0/1495429560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 msgr2=0x7f7d9006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.219+0000 7f7d8ed9d640 1 --2- 172.21.15.72:0/1495429560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 0x7f7d9006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.219+0000 7f7d8ed9d640 1 -- 172.21.15.72:0/1495429560 >> v1:172.21.15.72:6789/0 conn(0x7f7d900ce7e0 legacy=0x7f7d9015ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.219+0000 7f7d8ed9d640 1 -- 172.21.15.72:0/1495429560 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9006c6b0 con 0x7f7d9006a460 2023-12-12T09:54:14.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.219+0000 7f7d8d59a640 1 -- 172.21.15.72:0/1495429560 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543490263 0 0) 0x7f7d90071cd0 con 0x7f7d900ce7e0 2023-12-12T09:54:14.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.220+0000 7f7d8ed9d640 1 --2- 172.21.15.72:0/1495429560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d9006a460 0x7f7d9006a8f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009d90 tx=0x7f7d8802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128422c1f7fba6b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:14.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.220+0000 7f7d8d59a640 1 -- 172.21.15.72:0/1495429560 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8802cb30 con 0x7f7d9006a460 2023-12-12T09:54:14.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.220+0000 7f7d8d59a640 1 -- 172.21.15.72:0/1495429560 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8802cc90 con 0x7f7d9006a460 2023-12-12T09:54:14.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.220+0000 7f7d8d59a640 1 -- 172.21.15.72:0/1495429560 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88034440 con 0x7f7d9006a460 2023-12-12T09:54:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.221+0000 7f7d9483d640 1 -- 172.21.15.72:0/1495429560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d9006a460 msgr2=0x7f7d9006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.221+0000 7f7d9483d640 1 --2- 172.21.15.72:0/1495429560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d9006a460 0x7f7d9006a8f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009d90 tx=0x7f7d8802bd80 comp rx=0 tx=0).stop 2023-12-12T09:54:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.221+0000 7f7d9483d640 1 -- 172.21.15.72:0/1495429560 shutdown_connections 2023-12-12T09:54:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.221+0000 7f7d9483d640 1 --2- 172.21.15.72:0/1495429560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d9006a460 0x7f7d9006a8f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.221+0000 7f7d9483d640 1 --2- 172.21.15.72:0/1495429560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 0x7f7d9006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.221+0000 7f7d9483d640 1 -- 172.21.15.72:0/1495429560 >> 172.21.15.72:0/1495429560 conn(0x7f7d900cee70 msgr2=0x7f7d900cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.221+0000 7f7d9483d640 1 -- 172.21.15.72:0/1495429560 shutdown_connections 2023-12-12T09:54:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.221+0000 7f7d9483d640 1 -- 172.21.15.72:0/1495429560 wait complete. 2023-12-12T09:54:14.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.223+0000 7f7d9483d640 1 Processor -- start 2023-12-12T09:54:14.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.223+0000 7f7d9483d640 1 -- start start 2023-12-12T09:54:14.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d9483d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d9006a460 0x7f7d90159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d9483d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d900ce7e0 0x7f7d900b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d9483d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 0x7f7d901f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d9483d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9015a190 con 0x7f7d900ce7e0 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d9483d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900c9d00 con 0x7f7d9006a460 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d9483d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d901f6d80 con 0x7f7d9015af50 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d8dd9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d900ce7e0 0x7f7d900b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d8dd9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d900ce7e0 0x7f7d900b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46728/0 (socket says 172.21.15.72:46728) 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.224+0000 7f7d8dd9b640 1 -- 172.21.15.72:0/2035220838 learned_addr learned my addr 172.21.15.72:0/2035220838 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8ed9d640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 0x7f7d901f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8e59c640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d9006a460 0x7f7d90159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8dd9b640 1 -- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 msgr2=0x7f7d901f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8dd9b640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 0x7f7d901f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8dd9b640 1 -- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d9006a460 msgr2=0x7f7d90159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8dd9b640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d9006a460 0x7f7d90159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8dd9b640 1 -- 172.21.15.72:0/2035220838 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d88009a70 con 0x7f7d900ce7e0 2023-12-12T09:54:14.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8e59c640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d9006a460 0x7f7d90159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:14.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d8dd9b640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d900ce7e0 0x7f7d900b1930 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400c430 tx=0x7f7d8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:14.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d7effd640 1 -- 172.21.15.72:0/2035220838 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d840074e0 con 0x7f7d900ce7e0 2023-12-12T09:54:14.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d7effd640 1 -- 172.21.15.72:0/2035220838 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d84007660 con 0x7f7d900ce7e0 2023-12-12T09:54:14.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d7effd640 1 -- 172.21.15.72:0/2035220838 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84004a20 con 0x7f7d900ce7e0 2023-12-12T09:54:14.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.225+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d901f7060 con 0x7f7d900ce7e0 2023-12-12T09:54:14.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.226+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d900ca230 con 0x7f7d900ce7e0 2023-12-12T09:54:14.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.227+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d900ca450 con 0x7f7d900ce7e0 2023-12-12T09:54:14.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.228+0000 7f7d7effd640 1 -- 172.21.15.72:0/2035220838 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d84004b80 con 0x7f7d900ce7e0 2023-12-12T09:54:14.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.229+0000 7f7d7effd640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d7006ae40 0x7f7d7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.229+0000 7f7d7effd640 1 -- 172.21.15.72:0/2035220838 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7d840895a0 con 0x7f7d900ce7e0 2023-12-12T09:54:14.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.229+0000 7f7d8e59c640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d7006ae40 0x7f7d7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.230+0000 7f7d8e59c640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d7006ae40 0x7f7d7006d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009ac0 tx=0x7f7d80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:14.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.234+0000 7f7d7effd640 1 -- 172.21.15.72:0/2035220838 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d840544f0 con 0x7f7d900ce7e0 2023-12-12T09:54:14.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.427+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d900722d0 con 0x7f7d900ce7e0 2023-12-12T09:54:14.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.427+0000 7f7d7effd640 1 -- 172.21.15.72:0/2035220838 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d84057b10 con 0x7f7d900ce7e0 2023-12-12T09:54:14.429 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:54:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d7006ae40 msgr2=0x7f7d7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d7006ae40 0x7f7d7006d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009ac0 tx=0x7f7d80008040 comp rx=0 tx=0).stop 2023-12-12T09:54:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d900ce7e0 msgr2=0x7f7d900b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d900ce7e0 0x7f7d900b1930 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400c430 tx=0x7f7d8400c8f0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 shutdown_connections 2023-12-12T09:54:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d7006ae40 0x7f7d7006d300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d9015af50 0x7f7d901f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d900ce7e0 0x7f7d900b1930 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 --2- 172.21.15.72:0/2035220838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d9006a460 0x7f7d90159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 >> 172.21.15.72:0/2035220838 conn(0x7f7d900cee70 msgr2=0x7f7d9015c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.430+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 shutdown_connections 2023-12-12T09:54:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.431+0000 7f7d9483d640 1 -- 172.21.15.72:0/2035220838 wait complete. 2023-12-12T09:54:14.453 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:54:14.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.660+0000 7fa05c0ad640 1 Processor -- start 2023-12-12T09:54:14.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.660+0000 7fa05c0ad640 1 -- start start 2023-12-12T09:54:14.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa05c0ad640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0540b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa05c0ad640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa05406c410 0x7fa05415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa05c0ad640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540b3bb0 con 0x7fa054143e00 2023-12-12T09:54:14.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa05c0ad640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0541441e0 con 0x7fa05406c410 2023-12-12T09:54:14.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa05c0ad640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa054072680 con 0x7fa054072180 2023-12-12T09:54:14.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa059648640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0540b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa05a64a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa05406c410 0x7fa05415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa059648640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0540b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:45654/0 (socket says 172.21.15.72:45654) 2023-12-12T09:54:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa05a64a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa05406c410 0x7fa05415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:37214/0 (socket says 172.21.15.72:37214) 2023-12-12T09:54:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa059648640 1 -- 172.21.15.72:0/3583763896 learned_addr learned my addr 172.21.15.72:0/3583763896 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa059648640 1 -- 172.21.15.72:0/3583763896 >> v2:172.21.15.113:3300/0 conn(0x7fa05406c410 msgr2=0x7fa05415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa059648640 1 --2- 172.21.15.72:0/3583763896 >> v2:172.21.15.113:3300/0 conn(0x7fa05406c410 0x7fa05415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa059648640 1 -- 172.21.15.72:0/3583763896 >> v1:172.21.15.113:6789/0 conn(0x7fa054143e00 legacy=0x7fa0540b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa059648640 1 -- 172.21.15.72:0/3583763896 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540b1e30 con 0x7fa054072180 2023-12-12T09:54:14.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.662+0000 7fa05a64a640 1 --2- 172.21.15.72:0/3583763896 >> v2:172.21.15.113:3300/0 conn(0x7fa05406c410 0x7fa05415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:54:14.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.664+0000 7fa059648640 1 --2- 172.21.15.72:0/3583763896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0540b4750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa044009870 tx=0x7fa04402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622989cfa188b16a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.664+0000 7fa058e47640 1 -- 172.21.15.72:0/3583763896 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044030840 con 0x7fa054072180 2023-12-12T09:54:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.664+0000 7fa058e47640 1 -- 172.21.15.72:0/3583763896 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0440309a0 con 0x7fa054072180 2023-12-12T09:54:14.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.664+0000 7fa058e47640 1 -- 172.21.15.72:0/3583763896 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044030d30 con 0x7fa054072180 2023-12-12T09:54:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.665+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3583763896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 msgr2=0x7fa0540b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.665+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3583763896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0540b4750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa044009870 tx=0x7fa04402b980 comp rx=0 tx=0).stop 2023-12-12T09:54:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.665+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3583763896 shutdown_connections 2023-12-12T09:54:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.665+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3583763896 >> v2:172.21.15.113:3300/0 conn(0x7fa05406c410 0x7fa05415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.665+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3583763896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0540b4750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.665+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3583763896 >> 172.21.15.72:0/3583763896 conn(0x7fa0540b0a90 msgr2=0x7fa0540b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.665+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3583763896 shutdown_connections 2023-12-12T09:54:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.665+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3583763896 wait complete. 2023-12-12T09:54:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.667+0000 7fa05c0ad640 1 Processor -- start 2023-12-12T09:54:14.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.667+0000 7fa05c0ad640 1 -- start start 2023-12-12T09:54:14.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.668+0000 7fa05c0ad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05406c410 0x7fa0541428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.668+0000 7fa05c0ad640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0541f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.668+0000 7fa05c0ad640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa054143e00 0x7fa054136a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa05c0ad640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05413ba90 con 0x7fa054143e00 2023-12-12T09:54:14.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa05c0ad640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05413ae10 con 0x7fa05406c410 2023-12-12T09:54:14.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa05c0ad640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05413b0f0 con 0x7fa054072180 2023-12-12T09:54:14.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa05a64a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa054143e00 0x7fa054136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa05a64a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa054143e00 0x7fa054136a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46734/0 (socket says 172.21.15.72:46734) 2023-12-12T09:54:14.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa05a64a640 1 -- 172.21.15.72:0/3714143927 learned_addr learned my addr 172.21.15.72:0/3714143927 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:14.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa059e49640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05406c410 0x7fa0541428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa059648640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0541f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa05a64a640 1 -- 172.21.15.72:0/3714143927 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 msgr2=0x7fa0541f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.669+0000 7fa05a64a640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0541f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa05a64a640 1 -- 172.21.15.72:0/3714143927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05406c410 msgr2=0x7fa0541428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa05a64a640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05406c410 0x7fa0541428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa05a64a640 1 -- 172.21.15.72:0/3714143927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c009a70 con 0x7fa054143e00 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa05a64a640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa054143e00 0x7fa054136a30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00efc0 tx=0x7fa04c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa04a7fc640 1 -- 172.21.15.72:0/3714143927 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c009d70 con 0x7fa054143e00 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa04a7fc640 1 -- 172.21.15.72:0/3714143927 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c00f040 con 0x7fa054143e00 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0440094d0 con 0x7fa054143e00 2023-12-12T09:54:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa04a7fc640 1 -- 172.21.15.72:0/3714143927 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c003990 con 0x7fa054143e00 2023-12-12T09:54:14.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.670+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05413b6b0 con 0x7fa054143e00 2023-12-12T09:54:14.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.672+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05413bbd0 con 0x7fa054143e00 2023-12-12T09:54:14.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.672+0000 7fa04a7fc640 1 -- 172.21.15.72:0/3714143927 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa04c0144a0 con 0x7fa054143e00 2023-12-12T09:54:14.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.673+0000 7fa04a7fc640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa03806ae40 0x7fa03806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:14.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.673+0000 7fa04a7fc640 1 -- 172.21.15.72:0/3714143927 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa04c08a3f0 con 0x7fa054143e00 2023-12-12T09:54:14.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.674+0000 7fa059e49640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa03806ae40 0x7fa03806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:14.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.674+0000 7fa059e49640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa03806ae40 0x7fa03806d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa050000f60 tx=0x7fa050009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:14.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.678+0000 7fa04a7fc640 1 -- 172.21.15.72:0/3714143927 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa05413bbd0 con 0x7fa054143e00 2023-12-12T09:54:14.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.882+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa05413bbd0 con 0x7fa054143e00 2023-12-12T09:54:14.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.883+0000 7fa04a7fc640 1 -- 172.21.15.72:0/3714143927 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa05413bbd0 con 0x7fa054143e00 2023-12-12T09:54:14.885 INFO:teuthology.orchestra.run.smithi072.stderr:listed 0 entries 2023-12-12T09:54:14.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa03806ae40 msgr2=0x7fa03806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa03806ae40 0x7fa03806d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa050000f60 tx=0x7fa050009040 comp rx=0 tx=0).stop 2023-12-12T09:54:14.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa054143e00 msgr2=0x7fa054136a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:14.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa054143e00 0x7fa054136a30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00efc0 tx=0x7fa04c009f90 comp rx=0 tx=0).stop 2023-12-12T09:54:14.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 shutdown_connections 2023-12-12T09:54:14.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa03806ae40 0x7fa03806d300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa054143e00 0x7fa054136a30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054072180 0x7fa0541f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 --2- 172.21.15.72:0/3714143927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05406c410 0x7fa0541428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:14.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 >> 172.21.15.72:0/3714143927 conn(0x7fa0540b0a90 msgr2=0x7fa0540cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:14.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 shutdown_connections 2023-12-12T09:54:14.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:14.886+0000 7fa05c0ad640 1 -- 172.21.15.72:0/3714143927 wait complete. 2023-12-12T09:54:14.906 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-12T09:54:14.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:14.906 DEBUG:teuthology.orchestra.run.smithi072:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-12T09:54:15.038 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:54:15.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:15.039 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-12T09:54:15.079 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:54:15.080 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:15.124 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054290,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-12T09:54:15.124 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:15.124 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:15.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054258,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-12T09:54:15.169 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:54:15.169 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:15.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054290,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-12T09:54:15.222 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:15.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.438+0000 7fd1d0fd7640 1 Processor -- start 2023-12-12T09:54:15.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.438+0000 7fd1d0fd7640 1 -- start start 2023-12-12T09:54:15.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1d0fd7640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd1cc143e00 0x7fd1cc0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1d0fd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 0x7fd1cc0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1d0fd7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc0b3bb0 con 0x7fd1cc06c410 2023-12-12T09:54:15.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1d0fd7640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc1441e0 con 0x7fd1cc143e00 2023-12-12T09:54:15.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1d0fd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc072680 con 0x7fd1cc072180 2023-12-12T09:54:15.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1ca59c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd1cc143e00 0x7fd1cc0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1ca59c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd1cc143e00 0x7fd1cc0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43890/0 (socket says 172.21.15.72:43890) 2023-12-12T09:54:15.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1ca59c640 1 -- 172.21.15.72:0/2621830257 learned_addr learned my addr 172.21.15.72:0/2621830257 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:15.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1c1d9b640 1 --2- 172.21.15.72:0/2621830257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 0x7fd1cc0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1ca59c640 1 -- 172.21.15.72:0/2621830257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 msgr2=0x7fd1cc0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:15.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1ca59c640 1 --2- 172.21.15.72:0/2621830257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 0x7fd1cc0b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1ca59c640 1 -- 172.21.15.72:0/2621830257 >> v1:172.21.15.113:6789/0 conn(0x7fd1cc06c410 legacy=0x7fd1cc070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:15.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.440+0000 7fd1ca59c640 1 -- 172.21.15.72:0/2621830257 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0b1e30 con 0x7fd1cc143e00 2023-12-12T09:54:15.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.441+0000 7fd1ca59c640 1 --2- 172.21.15.72:0/2621830257 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc0b4040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009e50 tx=0x7fd1bc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0195209bc8832a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:15.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.441+0000 7fd1c9d9b640 1 -- 172.21.15.72:0/2621830257 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc02ca70 con 0x7fd1cc143e00 2023-12-12T09:54:15.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.441+0000 7fd1c9d9b640 1 -- 172.21.15.72:0/2621830257 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc02cbd0 con 0x7fd1cc143e00 2023-12-12T09:54:15.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.441+0000 7fd1c9d9b640 1 -- 172.21.15.72:0/2621830257 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0343f0 con 0x7fd1cc143e00 2023-12-12T09:54:15.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.443+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2621830257 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 msgr2=0x7fd1cc0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.443+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2621830257 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc0b4040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009e50 tx=0x7fd1bc02bed0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.443+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2621830257 shutdown_connections 2023-12-12T09:54:15.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.443+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2621830257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 0x7fd1cc0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.443+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2621830257 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc0b4040 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.443+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2621830257 >> 172.21.15.72:0/2621830257 conn(0x7fd1cc0b0a90 msgr2=0x7fd1cc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:15.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.443+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2621830257 shutdown_connections 2023-12-12T09:54:15.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.443+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2621830257 wait complete. 2023-12-12T09:54:15.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.444+0000 7fd1d0fd7640 1 Processor -- start 2023-12-12T09:54:15.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.444+0000 7fd1d0fd7640 1 -- start start 2023-12-12T09:54:15.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1d0fd7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc06c410 0x7fd1cc15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1d0fd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 0x7fd1cc06cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1d0fd7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1d0fd7640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15b210 con 0x7fd1cc143e00 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1d0fd7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc1373b0 con 0x7fd1cc06c410 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1d0fd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc136310 con 0x7fd1cc072180 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1cad9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1cad9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc06d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43902/0 (socket says 172.21.15.72:43902) 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.445+0000 7fd1cad9d640 1 -- 172.21.15.72:0/2273649355 learned_addr learned my addr 172.21.15.72:0/2273649355 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1ca59c640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc06c410 0x7fd1cc15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1c1d9b640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 0x7fd1cc06cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1cad9d640 1 -- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 msgr2=0x7fd1cc06cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1cad9d640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 0x7fd1cc06cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1cad9d640 1 -- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc06c410 msgr2=0x7fd1cc15acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1cad9d640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc06c410 0x7fd1cc15acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1cad9d640 1 -- 172.21.15.72:0/2273649355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc009b30 con 0x7fd1cc143e00 2023-12-12T09:54:15.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1ca59c640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc06c410 0x7fd1cc15acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:15.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1cad9d640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc06d460 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b410 tx=0x7fd1c400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:15.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1c37fe640 1 -- 172.21.15.72:0/2273649355 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4004020 con 0x7fd1cc143e00 2023-12-12T09:54:15.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc1365f0 con 0x7fd1cc143e00 2023-12-12T09:54:15.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1c37fe640 1 -- 172.21.15.72:0/2273649355 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c4004180 con 0x7fd1cc143e00 2023-12-12T09:54:15.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.446+0000 7fd1c37fe640 1 -- 172.21.15.72:0/2273649355 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4011630 con 0x7fd1cc143e00 2023-12-12T09:54:15.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.447+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc136bb0 con 0x7fd1cc143e00 2023-12-12T09:54:15.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.448+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc0745d0 con 0x7fd1cc143e00 2023-12-12T09:54:15.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.449+0000 7fd1c37fe640 1 -- 172.21.15.72:0/2273649355 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1c4011790 con 0x7fd1cc143e00 2023-12-12T09:54:15.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.451+0000 7fd1c37fe640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1ac06ae70 0x7fd1ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.451+0000 7fd1c37fe640 1 -- 172.21.15.72:0/2273649355 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd1c40894a0 con 0x7fd1cc143e00 2023-12-12T09:54:15.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.451+0000 7fd1ca59c640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1ac06ae70 0x7fd1ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.451+0000 7fd1ca59c640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1ac06ae70 0x7fd1ac06d330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc035000 tx=0x7fd1bc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:15.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.454+0000 7fd1c37fe640 1 -- 172.21.15.72:0/2273649355 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1c4054360 con 0x7fd1cc143e00 2023-12-12T09:54:15.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.684+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1cc13f340 con 0x7fd1cc143e00 2023-12-12T09:54:15.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.685+0000 7fd1c37fe640 1 -- 172.21.15.72:0/2273649355 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fd1c4057980 con 0x7fd1cc143e00 2023-12-12T09:54:15.686 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:15.686 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:15.686 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:15.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.687+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1ac06ae70 msgr2=0x7fd1ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.687+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1ac06ae70 0x7fd1ac06d330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc035000 tx=0x7fd1bc030000 comp rx=0 tx=0).stop 2023-12-12T09:54:15.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.687+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 msgr2=0x7fd1cc06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.687+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc06d460 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b410 tx=0x7fd1c400b8d0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.688+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 shutdown_connections 2023-12-12T09:54:15.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.688+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1ac06ae70 0x7fd1ac06d330 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.688+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1cc143e00 0x7fd1cc06d460 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.688+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1cc072180 0x7fd1cc06cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.688+0000 7fd1d0fd7640 1 --2- 172.21.15.72:0/2273649355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1cc06c410 0x7fd1cc15acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.688+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 >> 172.21.15.72:0/2273649355 conn(0x7fd1cc0b0a90 msgr2=0x7fd1cc0afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:15.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.688+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 shutdown_connections 2023-12-12T09:54:15.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.688+0000 7fd1d0fd7640 1 -- 172.21.15.72:0/2273649355 wait complete. 2023-12-12T09:54:15.709 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:15.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.922+0000 7f1d8907c640 1 Processor -- start 2023-12-12T09:54:15.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.923+0000 7f1d8907c640 1 -- start start 2023-12-12T09:54:15.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.924+0000 7f1d8907c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1d84157a60 0x7f1d8406c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.924+0000 7f1d8907c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 0x7f1d8415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.924+0000 7f1d8907c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8406ca40 con 0x7f1d8415e0e0 2023-12-12T09:54:15.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.924+0000 7f1d8907c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8415e550 con 0x7f1d84157a60 2023-12-12T09:54:15.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.924+0000 7f1d8907c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84071cd0 con 0x7f1d8406de20 2023-12-12T09:54:15.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.925+0000 7f1d82d9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1d84157a60 0x7f1d8406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.925+0000 7f1d8359e640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1d8415e0e0 0x7f1d8415a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47262/0 (socket says 172.21.15.72:47262) 2023-12-12T09:54:15.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.925+0000 7f1d8359e640 1 -- 172.21.15.72:0/795262661 learned_addr learned my addr 172.21.15.72:0/795262661 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:15.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.925+0000 7f1d8259c640 1 --2- 172.21.15.72:0/795262661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 0x7f1d8415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.925+0000 7f1d82d9d640 1 -- 172.21.15.72:0/795262661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 msgr2=0x7f1d8415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:15.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.925+0000 7f1d82d9d640 1 --2- 172.21.15.72:0/795262661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 0x7f1d8415dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.925+0000 7f1d82d9d640 1 -- 172.21.15.72:0/795262661 >> v1:172.21.15.72:6789/0 conn(0x7f1d8415e0e0 legacy=0x7f1d8415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.925+0000 7f1d82d9d640 1 -- 172.21.15.72:0/795262661 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84070f10 con 0x7f1d84157a60 2023-12-12T09:54:15.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.926+0000 7f1d82d9d640 1 --2- 172.21.15.72:0/795262661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d8406c410 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0097a0 tx=0x7f1d6c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4ee340791be79 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:15.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d81d9b640 1 -- 172.21.15.72:0/795262661 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c031780 con 0x7f1d84157a60 2023-12-12T09:54:15.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d81d9b640 1 -- 172.21.15.72:0/795262661 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c0318e0 con 0x7f1d84157a60 2023-12-12T09:54:15.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d81d9b640 1 -- 172.21.15.72:0/795262661 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c031c70 con 0x7f1d84157a60 2023-12-12T09:54:15.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d8907c640 1 -- 172.21.15.72:0/795262661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 msgr2=0x7f1d8406c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d8907c640 1 --2- 172.21.15.72:0/795262661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d8406c410 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0097a0 tx=0x7f1d6c02b820 comp rx=0 tx=0).stop 2023-12-12T09:54:15.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d8907c640 1 -- 172.21.15.72:0/795262661 shutdown_connections 2023-12-12T09:54:15.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d8907c640 1 --2- 172.21.15.72:0/795262661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 0x7f1d8415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d8907c640 1 --2- 172.21.15.72:0/795262661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d8406c410 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.927+0000 7f1d8907c640 1 -- 172.21.15.72:0/795262661 >> 172.21.15.72:0/795262661 conn(0x7f1d840b0a90 msgr2=0x7f1d840b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:15.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.928+0000 7f1d8907c640 1 -- 172.21.15.72:0/795262661 shutdown_connections 2023-12-12T09:54:15.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.928+0000 7f1d8907c640 1 -- 172.21.15.72:0/795262661 wait complete. 2023-12-12T09:54:15.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.929+0000 7f1d8907c640 1 Processor -- start 2023-12-12T09:54:15.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.929+0000 7f1d8907c640 1 -- start start 2023-12-12T09:54:15.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.930+0000 7f1d8907c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 0x7f1d8414a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.930+0000 7f1d8907c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d840b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.930+0000 7f1d8907c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d8415e0e0 0x7f1d840b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.930+0000 7f1d8907c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84158e60 con 0x7f1d84157a60 2023-12-12T09:54:15.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.930+0000 7f1d8907c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84159140 con 0x7f1d8415e0e0 2023-12-12T09:54:15.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.930+0000 7f1d8907c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84159420 con 0x7f1d8406de20 2023-12-12T09:54:15.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d840b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d840b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43930/0 (socket says 172.21.15.72:43930) 2023-12-12T09:54:15.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 -- 172.21.15.72:0/2945736174 learned_addr learned my addr 172.21.15.72:0/2945736174 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d82d9d640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 0x7f1d8414a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8359e640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d8415e0e0 0x7f1d840b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 -- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 msgr2=0x7f1d8414a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 0x7f1d8414a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 -- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d8415e0e0 msgr2=0x7f1d840b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d8415e0e0 0x7f1d840b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 -- 172.21.15.72:0/2945736174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c009400 con 0x7f1d84157a60 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8359e640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d8415e0e0 0x7f1d840b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:15.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.931+0000 7f1d8259c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d840b39b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400d800 tx=0x7f1d7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:15.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.932+0000 7f1d7b7fe640 1 -- 172.21.15.72:0/2945736174 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7400b6c0 con 0x7f1d84157a60 2023-12-12T09:54:15.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.932+0000 7f1d7b7fe640 1 -- 172.21.15.72:0/2945736174 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7400b820 con 0x7f1d84157a60 2023-12-12T09:54:15.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.932+0000 7f1d7b7fe640 1 -- 172.21.15.72:0/2945736174 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7400bbb0 con 0x7f1d84157a60 2023-12-12T09:54:15.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.932+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d84159700 con 0x7f1d84157a60 2023-12-12T09:54:15.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.932+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8414e7b0 con 0x7f1d84157a60 2023-12-12T09:54:15.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.933+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d840745d0 con 0x7f1d84157a60 2023-12-12T09:54:15.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.934+0000 7f1d7b7fe640 1 -- 172.21.15.72:0/2945736174 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d7400bd10 con 0x7f1d84157a60 2023-12-12T09:54:15.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.935+0000 7f1d7b7fe640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1d6406ae40 0x7f1d6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:15.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.935+0000 7f1d7b7fe640 1 -- 172.21.15.72:0/2945736174 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1d74089d20 con 0x7f1d84157a60 2023-12-12T09:54:15.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.935+0000 7f1d82d9d640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1d6406ae40 0x7f1d6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:15.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.936+0000 7f1d82d9d640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1d6406ae40 0x7f1d6406d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c036000 tx=0x7f1d6c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:15.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:15.939+0000 7f1d7b7fe640 1 -- 172.21.15.72:0/2945736174 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d74054c60 con 0x7f1d84157a60 2023-12-12T09:54:16.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.129+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d841547d0 con 0x7f1d84157a60 2023-12-12T09:54:16.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.131+0000 7f1d7b7fe640 1 -- 172.21.15.72:0/2945736174 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16823 (secure 0 0 0) 0x7f1d74058280 con 0x7f1d84157a60 2023-12-12T09:54:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:16.133 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":40,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:53:56.442722+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:53:45.083566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-12T09:53:46.274070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:16.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1d6406ae40 msgr2=0x7f1d6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1d6406ae40 0x7f1d6406d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c036000 tx=0x7f1d6c03d040 comp rx=0 tx=0).stop 2023-12-12T09:54:16.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 msgr2=0x7f1d840b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d840b39b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400d800 tx=0x7f1d7400dcc0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 shutdown_connections 2023-12-12T09:54:16.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1d6406ae40 0x7f1d6406d300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d8415e0e0 0x7f1d840b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d84157a60 0x7f1d840b39b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 --2- 172.21.15.72:0/2945736174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d8406de20 0x7f1d8414a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 >> 172.21.15.72:0/2945736174 conn(0x7f1d840b0a90 msgr2=0x7f1d840ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:16.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 shutdown_connections 2023-12-12T09:54:16.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.134+0000 7f1d8907c640 1 -- 172.21.15.72:0/2945736174 wait complete. 2023-12-12T09:54:16.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:53:45.083566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:53:46.274070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:54:16.155 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:54:16.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.365+0000 7f9a33d8a640 1 Processor -- start 2023-12-12T09:54:16.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.365+0000 7f9a33d8a640 1 -- start start 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.366+0000 7f9a33d8a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a2c0ae300 0x7f9a2c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.366+0000 7f9a33d8a640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2c0cdaa0 con 0x7f9a2c15a1e0 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.366+0000 7f9a33d8a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2c0cdd80 con 0x7f9a2c06ab70 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.366+0000 7f9a33d8a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2c0ce060 con 0x7f9a2c0ae300 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.367+0000 7f9a31b26640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f9a2c15a1e0 0x7f9a2c06a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47278/0 (socket says 172.21.15.72:47278) 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.367+0000 7f9a31b26640 1 -- 172.21.15.72:0/1360988819 learned_addr learned my addr 172.21.15.72:0/1360988819 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.367+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998743905 0 0) 0x7f9a2c0cdaa0 con 0x7f9a2c15a1e0 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.367+0000 7f9a32327640 1 --2- 172.21.15.72:0/1360988819 >> v2:172.21.15.113:3300/0 conn(0x7f9a2c0ae300 0x7f9a2c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.367+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a18005730 con 0x7f9a2c15a1e0 2023-12-12T09:54:16.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.367+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572577544 0 0) 0x7f9a18005730 con 0x7f9a2c15a1e0 2023-12-12T09:54:16.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 >> v2:172.21.15.113:3300/0 conn(0x7f9a2c0ae300 msgr2=0x7f9a2c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:16.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 --2- 172.21.15.72:0/1360988819 >> v2:172.21.15.113:3300/0 conn(0x7f9a2c0ae300 0x7f9a2c06ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 >> v1:172.21.15.113:6789/0 conn(0x7f9a2c06ab70 legacy=0x7f9a2c15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c0ce340 con 0x7f9a2c15a1e0 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f9a28001670 con 0x7f9a2c15a1e0 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 >> v1:172.21.15.72:6789/0 conn(0x7f9a2c15a1e0 legacy=0x7f9a2c06a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a18005b80 0x7f9a18005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a18006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a180074b0 0x7f9a1800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c0ce060 con 0x7f9a180069d0 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1800e380 con 0x7f9a180074b0 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1800e540 con 0x7f9a18005b80 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a31325640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a18006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a31325640 1 -- 172.21.15.72:0/1360988819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a18005b80 msgr2=0x7f9a18005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a31b26640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a18005b80 0x7f9a18005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a31325640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a18005b80 0x7f9a18005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a31325640 1 -- 172.21.15.72:0/1360988819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a180074b0 msgr2=0x7f9a1800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a31325640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a180074b0 0x7f9a1800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a31325640 1 -- 172.21.15.72:0/1360988819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a18005730 con 0x7f9a180069d0 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.368+0000 7f9a32327640 1 -- 172.21.15.72:0/1360988819 reap_dead start 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.370+0000 7f9a31325640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a18006e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c001720 tx=0x7f9a18007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d815fbe13642a1d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.370+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0052d0 con 0x7f9a180069d0 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.370+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c02e050 con 0x7f9a180069d0 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.370+0000 7f9a30b24640 1 -- 172.21.15.72:0/1360988819 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c003830 con 0x7f9a180069d0 2023-12-12T09:54:16.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.370+0000 7f9a33d8a640 1 -- 172.21.15.72:0/1360988819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 msgr2=0x7f9a18006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.370+0000 7f9a33d8a640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a18006e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c001720 tx=0x7f9a18007a00 comp rx=0 tx=0).stop 2023-12-12T09:54:16.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.371+0000 7f9a33d8a640 1 -- 172.21.15.72:0/1360988819 shutdown_connections 2023-12-12T09:54:16.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.371+0000 7f9a33d8a640 1 --2- 172.21.15.72:0/1360988819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a18006e50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.371+0000 7f9a33d8a640 1 -- 172.21.15.72:0/1360988819 >> 172.21.15.72:0/1360988819 conn(0x7f9a2c0cee70 msgr2=0x7f9a2c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:16.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.371+0000 7f9a33d8a640 1 -- 172.21.15.72:0/1360988819 shutdown_connections 2023-12-12T09:54:16.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.371+0000 7f9a33d8a640 1 -- 172.21.15.72:0/1360988819 wait complete. 2023-12-12T09:54:16.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.372+0000 7f9a33d8a640 1 Processor -- start 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.372+0000 7f9a33d8a640 1 -- start start 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.373+0000 7f9a33d8a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a2c0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.373+0000 7f9a33d8a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a2c15c020 0x7f9a2c099a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.373+0000 7f9a33d8a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2c099fc0 0x7f9a2c148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.373+0000 7f9a33d8a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c146c10 con 0x7f9a180069d0 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.373+0000 7f9a33d8a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c15c570 con 0x7f9a2c099fc0 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.373+0000 7f9a33d8a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c15c850 con 0x7f9a2c15c020 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a2c0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a2c0b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43954/0 (socket says 172.21.15.72:43954) 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 -- 172.21.15.72:0/3758639541 learned_addr learned my addr 172.21.15.72:0/3758639541 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:16.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31325640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a2c15c020 0x7f9a2c099a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 -- 172.21.15.72:0/3758639541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a2c15c020 msgr2=0x7f9a2c099a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a2c15c020 0x7f9a2c099a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 -- 172.21.15.72:0/3758639541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2c099fc0 msgr2=0x7f9a2c148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a32327640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2c099fc0 0x7f9a2c148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2c099fc0 0x7f9a2c148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 -- 172.21.15.72:0/3758639541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c15cb30 con 0x7f9a180069d0 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.374+0000 7f9a31b26640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a2c0b45c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800d770 tx=0x7f9a2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.375+0000 7f9a227fc640 1 -- 172.21.15.72:0/3758639541 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a280074e0 con 0x7f9a180069d0 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.375+0000 7f9a227fc640 1 -- 172.21.15.72:0/3758639541 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28007640 con 0x7f9a180069d0 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.375+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c15ce10 con 0x7f9a180069d0 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.375+0000 7f9a227fc640 1 -- 172.21.15.72:0/3758639541 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a280049e0 con 0x7f9a180069d0 2023-12-12T09:54:16.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.375+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c149300 con 0x7f9a180069d0 2023-12-12T09:54:16.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.376+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2c149580 con 0x7f9a180069d0 2023-12-12T09:54:16.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.377+0000 7f9a227fc640 1 -- 172.21.15.72:0/3758639541 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9a28004b40 con 0x7f9a180069d0 2023-12-12T09:54:16.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.378+0000 7f9a227fc640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9a1006ae70 0x7f9a1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.378+0000 7f9a227fc640 1 -- 172.21.15.72:0/3758639541 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9a280892b0 con 0x7f9a180069d0 2023-12-12T09:54:16.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.378+0000 7f9a31325640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9a1006ae70 0x7f9a1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.379+0000 7f9a31325640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9a1006ae70 0x7f9a1006d330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c033000 tx=0x7f9a1c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:16.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.382+0000 7f9a227fc640 1 -- 172.21.15.72:0/3758639541 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a28054210 con 0x7f9a180069d0 2023-12-12T09:54:16.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.575+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a2c074450 con 0x7f9a180069d0 2023-12-12T09:54:16.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.576+0000 7f9a227fc640 1 -- 172.21.15.72:0/3758639541 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a28057830 con 0x7f9a180069d0 2023-12-12T09:54:16.579 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:54:16.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9a1006ae70 msgr2=0x7f9a1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9a1006ae70 0x7f9a1006d330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c033000 tx=0x7f9a1c009030 comp rx=0 tx=0).stop 2023-12-12T09:54:16.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 msgr2=0x7f9a2c0b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a2c0b45c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800d770 tx=0x7f9a2800dc30 comp rx=0 tx=0).stop 2023-12-12T09:54:16.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 shutdown_connections 2023-12-12T09:54:16.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f9a1006ae70 0x7f9a1006d330 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a2c099fc0 0x7f9a2c148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a2c15c020 0x7f9a2c099a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 --2- 172.21.15.72:0/3758639541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a180069d0 0x7f9a2c0b45c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 >> 172.21.15.72:0/3758639541 conn(0x7f9a2c0cee70 msgr2=0x7f9a2c0ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 shutdown_connections 2023-12-12T09:54:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.579+0000 7f9a33d8a640 1 -- 172.21.15.72:0/3758639541 wait complete. 2023-12-12T09:54:16.600 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:54:16.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.808+0000 7fce47fff640 1 Processor -- start 2023-12-12T09:54:16.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.809+0000 7fce47fff640 1 -- start start 2023-12-12T09:54:16.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.810+0000 7fce47fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce48071c70 0x7fce48165ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.810+0000 7fce47fff640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4806c770 con 0x7fce480cb5b0 2023-12-12T09:54:16.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.810+0000 7fce47fff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4806ca50 con 0x7fce4806c130 2023-12-12T09:54:16.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.810+0000 7fce47fff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4806cd30 con 0x7fce48071c70 2023-12-12T09:54:16.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.810+0000 7fce46ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fce480cb5b0 0x7fce480cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47294/0 (socket says 172.21.15.72:47294) 2023-12-12T09:54:16.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce46ffd640 1 -- 172.21.15.72:0/3875751435 learned_addr learned my addr 172.21.15.72:0/3875751435 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:16.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238232467 0 0) 0x7fce4806c770 con 0x7fce480cb5b0 2023-12-12T09:54:16.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce34003520 con 0x7fce480cb5b0 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce477fe640 1 --2- 172.21.15.72:0/3875751435 >> v2:172.21.15.113:3300/0 conn(0x7fce48071c70 0x7fce48165ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2536389715 0 0) 0x7fce34003520 con 0x7fce480cb5b0 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 >> v2:172.21.15.113:3300/0 conn(0x7fce48071c70 msgr2=0x7fce48165ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 --2- 172.21.15.72:0/3875751435 >> v2:172.21.15.113:3300/0 conn(0x7fce48071c70 0x7fce48165ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 >> v1:172.21.15.113:6789/0 conn(0x7fce4806c130 legacy=0x7fce4815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4806d010 con 0x7fce480cb5b0 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fce38001670 con 0x7fce480cb5b0 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 >> v1:172.21.15.72:6789/0 conn(0x7fce480cb5b0 legacy=0x7fce480cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce34003970 0x7fce34003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.811+0000 7fce467fc640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce34005180 0x7fce3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4806cd30 con 0x7fce340047c0 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3400c0b0 con 0x7fce34003970 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3400c270 con 0x7fce34005180 2023-12-12T09:54:16.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce3ffff640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce46ffd640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce34003970 0x7fce34003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce3ffff640 1 -- 172.21.15.72:0/3875751435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce34005180 msgr2=0x7fce3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:16.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce3ffff640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce34005180 0x7fce3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce3ffff640 1 -- 172.21.15.72:0/3875751435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce34003970 msgr2=0x7fce34003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:16.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce3ffff640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce34003970 0x7fce34003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce3ffff640 1 -- 172.21.15.72:0/3875751435 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34003520 con 0x7fce340047c0 2023-12-12T09:54:16.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.812+0000 7fce477fe640 1 -- 172.21.15.72:0/3875751435 reap_dead start 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.813+0000 7fce3ffff640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce34004c40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fce480cfc70 tx=0x7fce340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23a6c50aa53e8e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.814+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30032600 con 0x7fce340047c0 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.814+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3002e050 con 0x7fce340047c0 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.814+0000 7fce467fc640 1 -- 172.21.15.72:0/3875751435 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce300329e0 con 0x7fce340047c0 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.814+0000 7fce47fff640 1 -- 172.21.15.72:0/3875751435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 msgr2=0x7fce34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.814+0000 7fce47fff640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce34004c40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fce480cfc70 tx=0x7fce340057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.814+0000 7fce47fff640 1 -- 172.21.15.72:0/3875751435 shutdown_connections 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.814+0000 7fce47fff640 1 --2- 172.21.15.72:0/3875751435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce34004c40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.814+0000 7fce47fff640 1 -- 172.21.15.72:0/3875751435 >> 172.21.15.72:0/3875751435 conn(0x7fce480b0940 msgr2=0x7fce480b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:16.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.815+0000 7fce47fff640 1 -- 172.21.15.72:0/3875751435 shutdown_connections 2023-12-12T09:54:16.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.815+0000 7fce47fff640 1 -- 172.21.15.72:0/3875751435 wait complete. 2023-12-12T09:54:16.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.816+0000 7fce47fff640 1 Processor -- start 2023-12-12T09:54:16.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.817+0000 7fce47fff640 1 -- start start 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.817+0000 7fce47fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce480cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.817+0000 7fce47fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce481499f0 0x7fce481f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce47fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce480af2e0 0x7fce480af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce47fff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480ae240 con 0x7fce340047c0 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce47fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480ae520 con 0x7fce481499f0 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce47fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480ae800 con 0x7fce480af2e0 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce46ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce480cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce46ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce480cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43986/0 (socket says 172.21.15.72:43986) 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce46ffd640 1 -- 172.21.15.72:0/1077607571 learned_addr learned my addr 172.21.15.72:0/1077607571 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:16.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce46ffd640 1 -- 172.21.15.72:0/1077607571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce480af2e0 msgr2=0x7fce480af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce46ffd640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce480af2e0 0x7fce480af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce46ffd640 1 -- 172.21.15.72:0/1077607571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce481499f0 msgr2=0x7fce481f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce46ffd640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce481499f0 0x7fce481f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.818+0000 7fce46ffd640 1 -- 172.21.15.72:0/1077607571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480aeae0 con 0x7fce340047c0 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.819+0000 7fce46ffd640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce480cac10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fce3800e840 tx=0x7fce3800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.819+0000 7fce3f7fe640 1 -- 172.21.15.72:0/1077607571 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3800c8d0 con 0x7fce340047c0 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.819+0000 7fce3f7fe640 1 -- 172.21.15.72:0/1077607571 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3800ca50 con 0x7fce340047c0 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.819+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce480aedc0 con 0x7fce340047c0 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.819+0000 7fce3f7fe640 1 -- 172.21.15.72:0/1077607571 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3800ce00 con 0x7fce340047c0 2023-12-12T09:54:16.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.819+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce481f2ef0 con 0x7fce340047c0 2023-12-12T09:54:16.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.820+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce48067910 con 0x7fce340047c0 2023-12-12T09:54:16.823 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.821+0000 7fce3f7fe640 1 -- 172.21.15.72:0/1077607571 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce3801d460 con 0x7fce340047c0 2023-12-12T09:54:16.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.822+0000 7fce3f7fe640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce2806ae40 0x7fce2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:16.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.823+0000 7fce3f7fe640 1 -- 172.21.15.72:0/1077607571 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce38089900 con 0x7fce340047c0 2023-12-12T09:54:16.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.823+0000 7fce3ffff640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce2806ae40 0x7fce2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:16.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.823+0000 7fce3ffff640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce2806ae40 0x7fce2806d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce30034000 tx=0x7fce3002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:16.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:16.827+0000 7fce3f7fe640 1 -- 172.21.15.72:0/1077607571 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce380547b0 con 0x7fce340047c0 2023-12-12T09:54:17.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.017+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce480af000 con 0x7fce340047c0 2023-12-12T09:54:17.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.018+0000 7fce3f7fe640 1 -- 172.21.15.72:0/1077607571 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce480af000 con 0x7fce340047c0 2023-12-12T09:54:17.020 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:54:17.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce2806ae40 msgr2=0x7fce2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce2806ae40 0x7fce2806d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce30034000 tx=0x7fce3002f000 comp rx=0 tx=0).stop 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 msgr2=0x7fce480cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce480cac10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fce3800e840 tx=0x7fce3800ed00 comp rx=0 tx=0).stop 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 shutdown_connections 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fce2806ae40 0x7fce2806d300 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce480af2e0 0x7fce480af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce481499f0 0x7fce481f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 --2- 172.21.15.72:0/1077607571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce340047c0 0x7fce480cac10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.020+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 >> 172.21.15.72:0/1077607571 conn(0x7fce480b0940 msgr2=0x7fce48157940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.021+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 shutdown_connections 2023-12-12T09:54:17.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.021+0000 7fce47fff640 1 -- 172.21.15.72:0/1077607571 wait complete. 2023-12-12T09:54:17.042 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:54:17.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.257+0000 7f4eeda3d640 1 Processor -- start 2023-12-12T09:54:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.258+0000 7f4eeda3d640 1 -- start start 2023-12-12T09:54:17.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.259+0000 7f4eeda3d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ee8072210 0x7f4ee8161530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.259+0000 7f4eeda3d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee8161a70 0x7f4ee80c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.259+0000 7f4eeda3d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee8160dc0 con 0x7f4ee815d0e0 2023-12-12T09:54:17.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.259+0000 7f4eeda3d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee80ceab0 con 0x7f4ee8072210 2023-12-12T09:54:17.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.259+0000 7f4eeda3d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8070280 con 0x7f4ee8161a70 2023-12-12T09:54:17.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.260+0000 7f4ee6ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4ee815d0e0 0x7f4ee80ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47310/0 (socket says 172.21.15.72:47310) 2023-12-12T09:54:17.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.260+0000 7f4ee6ffd640 1 -- 172.21.15.72:0/1761101854 learned_addr learned my addr 172.21.15.72:0/1761101854 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:17.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.260+0000 7f4ee77fe640 1 --2- 172.21.15.72:0/1761101854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee8161a70 0x7f4ee80c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.260+0000 7f4ede7fc640 1 --2- 172.21.15.72:0/1761101854 >> v2:172.21.15.113:3300/0 conn(0x7f4ee8072210 0x7f4ee8161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.261+0000 7f4ee67fc640 1 -- 172.21.15.72:0/1761101854 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334219583 0 0) 0x7f4ee8160dc0 con 0x7f4ee815d0e0 2023-12-12T09:54:17.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.261+0000 7f4ee67fc640 1 -- 172.21.15.72:0/1761101854 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed4003520 con 0x7f4ee815d0e0 2023-12-12T09:54:17.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.261+0000 7f4ee77fe640 1 -- 172.21.15.72:0/1761101854 >> v2:172.21.15.113:3300/0 conn(0x7f4ee8072210 msgr2=0x7f4ee8161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:17.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.261+0000 7f4ee77fe640 1 --2- 172.21.15.72:0/1761101854 >> v2:172.21.15.113:3300/0 conn(0x7f4ee8072210 0x7f4ee8161530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.261+0000 7f4ee67fc640 1 -- 172.21.15.72:0/1761101854 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1687404676 0 0) 0x7f4ed4003520 con 0x7f4ee815d0e0 2023-12-12T09:54:17.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.261+0000 7f4ee77fe640 1 -- 172.21.15.72:0/1761101854 >> v1:172.21.15.72:6789/0 conn(0x7f4ee815d0e0 legacy=0x7f4ee80ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.261+0000 7f4ee77fe640 1 -- 172.21.15.72:0/1761101854 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee8070560 con 0x7f4ee8161a70 2023-12-12T09:54:17.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.263+0000 7f4ee77fe640 1 --2- 172.21.15.72:0/1761101854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee8161a70 0x7f4ee80c92e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009d90 tx=0x7f4ee002bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ca1cc86327f905 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:17.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.263+0000 7f4ee67fc640 1 -- 172.21.15.72:0/1761101854 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8001670 con 0x7f4ee8161a70 2023-12-12T09:54:17.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.263+0000 7f4ee67fc640 1 -- 172.21.15.72:0/1761101854 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee002c8a0 con 0x7f4ee8161a70 2023-12-12T09:54:17.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.263+0000 7f4ee67fc640 1 -- 172.21.15.72:0/1761101854 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee002cc10 con 0x7f4ee8161a70 2023-12-12T09:54:17.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.264+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1761101854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee8161a70 msgr2=0x7f4ee80c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.264+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1761101854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee8161a70 0x7f4ee80c92e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009d90 tx=0x7f4ee002bd70 comp rx=0 tx=0).stop 2023-12-12T09:54:17.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.264+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1761101854 shutdown_connections 2023-12-12T09:54:17.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.264+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1761101854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee8161a70 0x7f4ee80c92e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.264+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1761101854 >> v2:172.21.15.113:3300/0 conn(0x7f4ee8072210 0x7f4ee8161530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.264+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1761101854 >> 172.21.15.72:0/1761101854 conn(0x7f4ee80cee70 msgr2=0x7f4ee80cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:17.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.264+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1761101854 shutdown_connections 2023-12-12T09:54:17.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.264+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1761101854 wait complete. 2023-12-12T09:54:17.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.266+0000 7f4eeda3d640 1 Processor -- start 2023-12-12T09:54:17.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.266+0000 7f4eeda3d640 1 -- start start 2023-12-12T09:54:17.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4eeda3d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ee8072210 0x7f4ee809c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4eeda3d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee815d0e0 0x7f4ee80b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4eeda3d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ee80b3760 0x7f4ee80c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4eeda3d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b4bb0 con 0x7f4ee8072210 2023-12-12T09:54:17.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4eeda3d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b3fa0 con 0x7f4ee80b3760 2023-12-12T09:54:17.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4eeda3d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b4280 con 0x7f4ee815d0e0 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee6ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ee8072210 0x7f4ee809c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee6ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ee8072210 0x7f4ee809c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43998/0 (socket says 172.21.15.72:43998) 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee6ffd640 1 -- 172.21.15.72:0/1355180919 learned_addr learned my addr 172.21.15.72:0/1355180919 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee6ffd640 1 -- 172.21.15.72:0/1355180919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee815d0e0 msgr2=0x7f4ee80b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee6ffd640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee815d0e0 0x7f4ee80b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee6ffd640 1 -- 172.21.15.72:0/1355180919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ee80b3760 msgr2=0x7f4ee80c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee77fe640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ee80b3760 0x7f4ee80c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee6ffd640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ee80b3760 0x7f4ee80c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee6ffd640 1 -- 172.21.15.72:0/1355180919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0009a70 con 0x7f4ee8072210 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.267+0000 7f4ee77fe640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ee80b3760 0x7f4ee80c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.268+0000 7f4ee6ffd640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ee8072210 0x7f4ee809c120 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c750 tx=0x7f4ed800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.268+0000 7f4ee4ff9640 1 -- 172.21.15.72:0/1355180919 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8005ad0 con 0x7f4ee8072210 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.268+0000 7f4ee4ff9640 1 -- 172.21.15.72:0/1355180919 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed8005c50 con 0x7f4ee8072210 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.268+0000 7f4ee4ff9640 1 -- 172.21.15.72:0/1355180919 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8007070 con 0x7f4ee8072210 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.268+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee80b4560 con 0x7f4ee8072210 2023-12-12T09:54:17.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.268+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee80b3c10 con 0x7f4ee8072210 2023-12-12T09:54:17.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.269+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee8074450 con 0x7f4ee8072210 2023-12-12T09:54:17.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.270+0000 7f4ee4ff9640 1 -- 172.21.15.72:0/1355180919 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ed80071d0 con 0x7f4ee8072210 2023-12-12T09:54:17.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.271+0000 7f4ee4ff9640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ec806ae40 0x7f4ec806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.271+0000 7f4ee4ff9640 1 -- 172.21.15.72:0/1355180919 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4ed80894c0 con 0x7f4ee8072210 2023-12-12T09:54:17.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.271+0000 7f4ede7fc640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ec806ae40 0x7f4ec806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.272+0000 7f4ede7fc640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ec806ae40 0x7f4ec806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006a90 tx=0x7f4ed0006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:17.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.276+0000 7f4ee4ff9640 1 -- 172.21.15.72:0/1355180919 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ed8054410 con 0x7f4ee8072210 2023-12-12T09:54:17.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.469+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ee8071430 con 0x7f4ee8072210 2023-12-12T09:54:17.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.470+0000 7f4ee4ff9640 1 -- 172.21.15.72:0/1355180919 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ed8057a30 con 0x7f4ee8072210 2023-12-12T09:54:17.471 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ec806ae40 msgr2=0x7f4ec806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ec806ae40 0x7f4ec806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006a90 tx=0x7f4ed0006a20 comp rx=0 tx=0).stop 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ee8072210 msgr2=0x7f4ee809c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ee8072210 0x7f4ee809c120 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c750 tx=0x7f4ed800cc10 comp rx=0 tx=0).stop 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 shutdown_connections 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ec806ae40 0x7f4ec806d300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ee80b3760 0x7f4ee80c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ee815d0e0 0x7f4ee80b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 --2- 172.21.15.72:0/1355180919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ee8072210 0x7f4ee809c120 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.472+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 >> 172.21.15.72:0/1355180919 conn(0x7f4ee80cee70 msgr2=0x7f4ee806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:17.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.473+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 shutdown_connections 2023-12-12T09:54:17.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.473+0000 7f4eeda3d640 1 -- 172.21.15.72:0/1355180919 wait complete. 2023-12-12T09:54:17.492 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:17.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.701+0000 7f190db63640 1 Processor -- start 2023-12-12T09:54:17.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.701+0000 7f190db63640 1 -- start start 2023-12-12T09:54:17.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.702+0000 7f190db63640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f190815e5d0 0x7f190806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.702+0000 7f190db63640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1908071f90 con 0x7f1908071260 2023-12-12T09:54:17.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.702+0000 7f190db63640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1908072270 con 0x7f19080ae200 2023-12-12T09:54:17.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.702+0000 7f190db63640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1908070070 con 0x7f190815e5d0 2023-12-12T09:54:17.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f190d362640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1908071260 0x7f19080cded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47326/0 (socket says 172.21.15.72:47326) 2023-12-12T09:54:17.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f190d362640 1 -- 172.21.15.72:0/1953893658 learned_addr learned my addr 172.21.15.72:0/1953893658 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f190cb61640 1 --2- 172.21.15.72:0/1953893658 >> v2:172.21.15.113:3300/0 conn(0x7f190815e5d0 0x7f190806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353871753 0 0) 0x7f1908071f90 con 0x7f1908071260 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f8003520 con 0x7f1908071260 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1116953576 0 0) 0x7f18f8003520 con 0x7f1908071260 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 >> v2:172.21.15.113:3300/0 conn(0x7f190815e5d0 msgr2=0x7f190806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f19077fe640 1 --2- 172.21.15.72:0/1953893658 >> v2:172.21.15.113:3300/0 conn(0x7f190815e5d0 0x7f190806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 >> v1:172.21.15.113:6789/0 conn(0x7f19080ae200 legacy=0x7f1908070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1908070350 con 0x7f1908071260 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.703+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f19000019f0 con 0x7f1908071260 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 >> v1:172.21.15.72:6789/0 conn(0x7f1908071260 legacy=0x7f19080cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f19077fe640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18f8003970 0x7f18f8003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f19077fe640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f18f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f19077fe640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18f80052a0 0x7f18f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908070070 con 0x7f18f80047c0 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f800c150 con 0x7f18f80052a0 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f800c310 con 0x7f18f8003970 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f1907fff640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f18f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f190d362640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18f80052a0 0x7f18f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f1907fff640 1 -- 172.21.15.72:0/1953893658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18f8003970 msgr2=0x7f18f8003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f1907fff640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18f8003970 0x7f18f8003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f1907fff640 1 -- 172.21.15.72:0/1953893658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18f80052a0 msgr2=0x7f18f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f1907fff640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18f80052a0 0x7f18f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f1907fff640 1 -- 172.21.15.72:0/1953893658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8003520 con 0x7f18f80047c0 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f190d362640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18f80052a0 0x7f18f800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:17.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.704+0000 7f190d362640 1 -- 172.21.15.72:0/1953893658 reap_dead start 2023-12-12T09:54:17.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f1907fff640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f18f8004c40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f18f4003d80 tx=0x7f18f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149642b4d7923e17 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:17.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4032600 con 0x7f18f80047c0 2023-12-12T09:54:17.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f402e050 con 0x7f18f80047c0 2023-12-12T09:54:17.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f19077fe640 1 -- 172.21.15.72:0/1953893658 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f40329e0 con 0x7f18f80047c0 2023-12-12T09:54:17.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f190db63640 1 -- 172.21.15.72:0/1953893658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 msgr2=0x7f18f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f190db63640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f18f8004c40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f18f4003d80 tx=0x7f18f80057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f190db63640 1 -- 172.21.15.72:0/1953893658 shutdown_connections 2023-12-12T09:54:17.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f190db63640 1 --2- 172.21.15.72:0/1953893658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f18f8004c40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.706+0000 7f190db63640 1 -- 172.21.15.72:0/1953893658 >> 172.21.15.72:0/1953893658 conn(0x7f19080b0860 msgr2=0x7f19080b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:17.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.707+0000 7f190db63640 1 -- 172.21.15.72:0/1953893658 shutdown_connections 2023-12-12T09:54:17.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.707+0000 7f190db63640 1 -- 172.21.15.72:0/1953893658 wait complete. 2023-12-12T09:54:17.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.708+0000 7f190db63640 1 Processor -- start 2023-12-12T09:54:17.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.708+0000 7f190db63640 1 -- start start 2023-12-12T09:54:17.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.709+0000 7f190db63640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f19081506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.709+0000 7f190db63640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f190815ecb0 0x7f190814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.709+0000 7f190db63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f190815f130 0x7f190814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.709+0000 7f190db63640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908150bf0 con 0x7f18f80047c0 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.709+0000 7f190db63640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190815d770 con 0x7f190815f130 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.709+0000 7f190db63640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19081f1680 con 0x7f190815ecb0 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f19081506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f19081506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44016/0 (socket says 172.21.15.72:44016) 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 -- 172.21.15.72:0/2519458633 learned_addr learned my addr 172.21.15.72:0/2519458633 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 -- 172.21.15.72:0/2519458633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f190815ecb0 msgr2=0x7f190814f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f190815ecb0 0x7f190814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 -- 172.21.15.72:0/2519458633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f190815f130 msgr2=0x7f190814f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f190815f130 0x7f190814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 -- 172.21.15.72:0/2519458633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190814fbf0 con 0x7f18f80047c0 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.710+0000 7f190cb61640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f19081506b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00d800 tx=0x7f18fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.711+0000 7f19057fa640 1 -- 172.21.15.72:0/2519458633 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00b6c0 con 0x7f18f80047c0 2023-12-12T09:54:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.711+0000 7f19057fa640 1 -- 172.21.15.72:0/2519458633 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc00b820 con 0x7f18f80047c0 2023-12-12T09:54:17.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.711+0000 7f19057fa640 1 -- 172.21.15.72:0/2519458633 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00bbd0 con 0x7f18f80047c0 2023-12-12T09:54:17.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.711+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190814fed0 con 0x7f18f80047c0 2023-12-12T09:54:17.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.711+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19081f1960 con 0x7f18f80047c0 2023-12-12T09:54:17.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.712+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1908074450 con 0x7f18f80047c0 2023-12-12T09:54:17.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.713+0000 7f19057fa640 1 -- 172.21.15.72:0/2519458633 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18fc00bd30 con 0x7f18f80047c0 2023-12-12T09:54:17.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.714+0000 7f19057fa640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f18ec06ae40 0x7f18ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:17.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.714+0000 7f19057fa640 1 -- 172.21.15.72:0/2519458633 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f18fc089da0 con 0x7f18f80047c0 2023-12-12T09:54:17.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.714+0000 7f1907fff640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f18ec06ae40 0x7f18ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:17.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.715+0000 7f1907fff640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f18ec06ae40 0x7f18ec06d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f18f4034000 tx=0x7f18f402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:17.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.718+0000 7f19057fa640 1 -- 172.21.15.72:0/2519458633 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18fc054ce0 con 0x7f18f80047c0 2023-12-12T09:54:17.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.948+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19080adfe0 con 0x7f18f80047c0 2023-12-12T09:54:17.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.949+0000 7f19057fa640 1 -- 172.21.15.72:0/2519458633 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f18fc058300 con 0x7f18f80047c0 2023-12-12T09:54:17.951 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:17.951 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:17.952 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:17.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f18ec06ae40 msgr2=0x7f18ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f18ec06ae40 0x7f18ec06d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f18f4034000 tx=0x7f18f402f000 comp rx=0 tx=0).stop 2023-12-12T09:54:17.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 msgr2=0x7f19081506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:17.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f19081506b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00d800 tx=0x7f18fc00dcc0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 shutdown_connections 2023-12-12T09:54:17.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f18ec06ae40 0x7f18ec06d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f190815f130 0x7f190814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f190815ecb0 0x7f190814f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 --2- 172.21.15.72:0/2519458633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f80047c0 0x7f19081506b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:17.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 >> 172.21.15.72:0/2519458633 conn(0x7f19080b0860 msgr2=0x7f19081593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:17.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 shutdown_connections 2023-12-12T09:54:17.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:17.952+0000 7f190db63640 1 -- 172.21.15.72:0/2519458633 wait complete. 2023-12-12T09:54:17.973 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:18.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.185+0000 7fe0f1a7c640 1 Processor -- start 2023-12-12T09:54:18.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.185+0000 7fe0f1a7c640 1 -- start start 2023-12-12T09:54:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.186+0000 7fe0f1a7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0ec072180 0x7fe0ec0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.186+0000 7fe0f1a7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec06c410 0x7fe0ec15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.186+0000 7fe0f1a7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ec0b3bb0 con 0x7fe0ec143e00 2023-12-12T09:54:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.186+0000 7fe0f1a7c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ec1441e0 con 0x7fe0ec072180 2023-12-12T09:54:18.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.186+0000 7fe0f1a7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec072680 con 0x7fe0ec06c410 2023-12-12T09:54:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.187+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec06c410 0x7fe0ec15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.187+0000 7fe0e3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0ec072180 0x7fe0ec0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.187+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec06c410 0x7fe0ec15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:55486/0 (socket says 172.21.15.72:55486) 2023-12-12T09:54:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.187+0000 7fe0e3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0ec072180 0x7fe0ec0b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:40278/0 (socket says 172.21.15.72:40278) 2023-12-12T09:54:18.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.187+0000 7fe0ebfff640 1 -- 172.21.15.72:0/1706753004 learned_addr learned my addr 172.21.15.72:0/1706753004 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:18.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.188+0000 7fe0ebfff640 1 -- 172.21.15.72:0/1706753004 >> v2:172.21.15.113:3300/0 conn(0x7fe0ec072180 msgr2=0x7fe0ec0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:18.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.188+0000 7fe0ebfff640 1 --2- 172.21.15.72:0/1706753004 >> v2:172.21.15.113:3300/0 conn(0x7fe0ec072180 0x7fe0ec0b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.188+0000 7fe0ebfff640 1 -- 172.21.15.72:0/1706753004 >> v1:172.21.15.113:6789/0 conn(0x7fe0ec143e00 legacy=0x7fe0ec0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.188+0000 7fe0ebfff640 1 -- 172.21.15.72:0/1706753004 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec0b1e30 con 0x7fe0ec06c410 2023-12-12T09:54:18.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.188+0000 7fe0e3fff640 1 --2- 172.21.15.72:0/1706753004 >> v2:172.21.15.113:3300/0 conn(0x7fe0ec072180 0x7fe0ec0b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:54:18.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.189+0000 7fe0ebfff640 1 --2- 172.21.15.72:0/1706753004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec06c410 0x7fe0ec15ecb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009e50 tx=0x7fe0e402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d9f07a2510b3dc server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:18.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.190+0000 7fe0eaffd640 1 -- 172.21.15.72:0/1706753004 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e402cb00 con 0x7fe0ec06c410 2023-12-12T09:54:18.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.190+0000 7fe0eaffd640 1 -- 172.21.15.72:0/1706753004 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e402cc60 con 0x7fe0ec06c410 2023-12-12T09:54:18.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.190+0000 7fe0eaffd640 1 -- 172.21.15.72:0/1706753004 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40343f0 con 0x7fe0ec06c410 2023-12-12T09:54:18.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.191+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/1706753004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec06c410 msgr2=0x7fe0ec15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.191+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/1706753004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec06c410 0x7fe0ec15ecb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009e50 tx=0x7fe0e402bf60 comp rx=0 tx=0).stop 2023-12-12T09:54:18.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.191+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/1706753004 shutdown_connections 2023-12-12T09:54:18.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.191+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/1706753004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec06c410 0x7fe0ec15ecb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.191+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/1706753004 >> v2:172.21.15.113:3300/0 conn(0x7fe0ec072180 0x7fe0ec0b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.191+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/1706753004 >> 172.21.15.72:0/1706753004 conn(0x7fe0ec0b0a90 msgr2=0x7fe0ec0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:18.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.191+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/1706753004 shutdown_connections 2023-12-12T09:54:18.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.191+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/1706753004 wait complete. 2023-12-12T09:54:18.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.193+0000 7fe0f1a7c640 1 Processor -- start 2023-12-12T09:54:18.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.193+0000 7fe0f1a7c640 1 -- start start 2023-12-12T09:54:18.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0f1a7c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0ec06c410 0x7fe0ec159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0f1a7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ec072180 0x7fe0ec13b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0f1a7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec143e00 0x7fe0ec0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0f1a7c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec13aed0 con 0x7fe0ec06c410 2023-12-12T09:54:18.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0f1a7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec0ad270 con 0x7fe0ec072180 2023-12-12T09:54:18.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0f1a7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec13b170 con 0x7fe0ec143e00 2023-12-12T09:54:18.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0ec06c410 0x7fe0ec159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0ec06c410 0x7fe0ec159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44018/0 (socket says 172.21.15.72:44018) 2023-12-12T09:54:18.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0eb7fe640 1 -- 172.21.15.72:0/393209176 learned_addr learned my addr 172.21.15.72:0/393209176 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:18.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0ebfff640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec143e00 0x7fe0ec0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.194+0000 7fe0e3fff640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ec072180 0x7fe0ec13b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0eb7fe640 1 -- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec143e00 msgr2=0x7fe0ec0ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0eb7fe640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec143e00 0x7fe0ec0ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0eb7fe640 1 -- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ec072180 msgr2=0x7fe0ec13b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0eb7fe640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ec072180 0x7fe0ec13b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0eb7fe640 1 -- 172.21.15.72:0/393209176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d4009400 con 0x7fe0ec06c410 2023-12-12T09:54:18.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0ebfff640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec143e00 0x7fe0ec0ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:18.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0eb7fe640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0ec06c410 0x7fe0ec159d60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c400 tx=0x7fe0dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:18.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0e8ff9640 1 -- 172.21.15.72:0/393209176 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0052d0 con 0x7fe0ec06c410 2023-12-12T09:54:18.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0e8ff9640 1 -- 172.21.15.72:0/393209176 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc005450 con 0x7fe0ec06c410 2023-12-12T09:54:18.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e4009b30 con 0x7fe0ec06c410 2023-12-12T09:54:18.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.195+0000 7fe0e8ff9640 1 -- 172.21.15.72:0/393209176 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc007260 con 0x7fe0ec06c410 2023-12-12T09:54:18.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.196+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec0ad510 con 0x7fe0ec06c410 2023-12-12T09:54:18.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.197+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ec0745d0 con 0x7fe0ec06c410 2023-12-12T09:54:18.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.198+0000 7fe0e8ff9640 1 -- 172.21.15.72:0/393209176 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe0dc0073c0 con 0x7fe0ec06c410 2023-12-12T09:54:18.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.198+0000 7fe0e8ff9640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe0cc06ae40 0x7fe0cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.198+0000 7fe0e8ff9640 1 -- 172.21.15.72:0/393209176 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe0dc088b70 con 0x7fe0ec06c410 2023-12-12T09:54:18.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.199+0000 7fe0e3fff640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe0cc06ae40 0x7fe0cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.199+0000 7fe0e3fff640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe0cc06ae40 0x7fe0cc06d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec13a480 tx=0x7fe0d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:18.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.203+0000 7fe0e8ff9640 1 -- 172.21.15.72:0/393209176 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0dc053ac0 con 0x7fe0ec06c410 2023-12-12T09:54:18.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.391+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0ec06e020 con 0x7fe0ec06c410 2023-12-12T09:54:18.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.393+0000 7fe0e8ff9640 1 -- 172.21.15.72:0/393209176 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16823 (secure 0 0 0) 0x7fe0dc080e60 con 0x7fe0ec06c410 2023-12-12T09:54:18.395 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:18.396 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":40,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:53:56.442722+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:53:45.083566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-12T09:53:46.274070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe0cc06ae40 msgr2=0x7fe0cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe0cc06ae40 0x7fe0cc06d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec13a480 tx=0x7fe0d4008040 comp rx=0 tx=0).stop 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0ec06c410 msgr2=0x7fe0ec159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0ec06c410 0x7fe0ec159d60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c400 tx=0x7fe0dc00c8c0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 shutdown_connections 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe0cc06ae40 0x7fe0cc06d300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0ec143e00 0x7fe0ec0ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0ec072180 0x7fe0ec13b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 --2- 172.21.15.72:0/393209176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0ec06c410 0x7fe0ec159d60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 >> 172.21.15.72:0/393209176 conn(0x7fe0ec0b0a90 msgr2=0x7fe0ec0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 shutdown_connections 2023-12-12T09:54:18.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.396+0000 7fe0f1a7c640 1 -- 172.21.15.72:0/393209176 wait complete. 2023-12-12T09:54:18.417 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-12T09:54:18.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:18.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.633+0000 7fd3d52d7640 1 Processor -- start 2023-12-12T09:54:18.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.633+0000 7fd3d52d7640 1 -- start start 2023-12-12T09:54:18.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.635+0000 7fd3d52d7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3d015e5d0 0x7fd3d006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.635+0000 7fd3d52d7640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd3d0071260 0x7fd3d00cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.635+0000 7fd3d52d7640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d0071f90 con 0x7fd3d00ae200 2023-12-12T09:54:18.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.635+0000 7fd3d52d7640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d0072270 con 0x7fd3d0071260 2023-12-12T09:54:18.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.635+0000 7fd3d52d7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d0070070 con 0x7fd3d015e5d0 2023-12-12T09:54:18.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3c77fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd3d00ae200 0x7fd3d0070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47330/0 (socket says 172.21.15.72:47330) 2023-12-12T09:54:18.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3c77fe640 1 -- 172.21.15.72:0/2348311122 learned_addr learned my addr 172.21.15.72:0/2348311122 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:18.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3d4ad6640 1 --2- 172.21.15.72:0/2348311122 >> v2:172.21.15.72:3300/0 conn(0x7fd3d0071260 0x7fd3d00cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3cffff640 1 --2- 172.21.15.72:0/2348311122 >> v2:172.21.15.113:3300/0 conn(0x7fd3d015e5d0 0x7fd3d006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3cf7fe640 1 -- 172.21.15.72:0/2348311122 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055761600 0 0) 0x7fd3d0071f90 con 0x7fd3d00ae200 2023-12-12T09:54:18.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3d4ad6640 1 -- 172.21.15.72:0/2348311122 >> v2:172.21.15.113:3300/0 conn(0x7fd3d015e5d0 msgr2=0x7fd3d006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:18.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3d4ad6640 1 --2- 172.21.15.72:0/2348311122 >> v2:172.21.15.113:3300/0 conn(0x7fd3d015e5d0 0x7fd3d006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3d4ad6640 1 -- 172.21.15.72:0/2348311122 >> v1:172.21.15.72:6789/0 conn(0x7fd3d00ae200 legacy=0x7fd3d0070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.636+0000 7fd3d4ad6640 1 -- 172.21.15.72:0/2348311122 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d0070350 con 0x7fd3d0071260 2023-12-12T09:54:18.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.637+0000 7fd3d4ad6640 1 --2- 172.21.15.72:0/2348311122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d00cded0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd3d006bb50 tx=0x7fd3c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8b6d7d6f558ebe server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:18.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.637+0000 7fd3cf7fe640 1 -- 172.21.15.72:0/2348311122 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c802e070 con 0x7fd3d0071260 2023-12-12T09:54:18.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.637+0000 7fd3cf7fe640 1 -- 172.21.15.72:0/2348311122 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c80315a0 con 0x7fd3d0071260 2023-12-12T09:54:18.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.637+0000 7fd3cf7fe640 1 -- 172.21.15.72:0/2348311122 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8031930 con 0x7fd3d0071260 2023-12-12T09:54:18.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.638+0000 7fd3d52d7640 1 -- 172.21.15.72:0/2348311122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 msgr2=0x7fd3d00cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.638+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/2348311122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d00cded0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd3d006bb50 tx=0x7fd3c802b840 comp rx=0 tx=0).stop 2023-12-12T09:54:18.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.638+0000 7fd3d52d7640 1 -- 172.21.15.72:0/2348311122 shutdown_connections 2023-12-12T09:54:18.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.638+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/2348311122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d00cded0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.638+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/2348311122 >> v2:172.21.15.113:3300/0 conn(0x7fd3d015e5d0 0x7fd3d006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.638+0000 7fd3d52d7640 1 -- 172.21.15.72:0/2348311122 >> 172.21.15.72:0/2348311122 conn(0x7fd3d00b0860 msgr2=0x7fd3d00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:18.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.638+0000 7fd3d52d7640 1 -- 172.21.15.72:0/2348311122 shutdown_connections 2023-12-12T09:54:18.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.638+0000 7fd3d52d7640 1 -- 172.21.15.72:0/2348311122 wait complete. 2023-12-12T09:54:18.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.640+0000 7fd3d52d7640 1 Processor -- start 2023-12-12T09:54:18.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.640+0000 7fd3d52d7640 1 -- start start 2023-12-12T09:54:18.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.641+0000 7fd3d52d7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d0150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.641+0000 7fd3d52d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d00ae200 0x7fd3d01526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.641+0000 7fd3d52d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d015e5d0 0x7fd3d00b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.641+0000 7fd3d52d7640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d01522a0 con 0x7fd3d0071260 2023-12-12T09:54:18.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.641+0000 7fd3d52d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00b27d0 con 0x7fd3d015e5d0 2023-12-12T09:54:18.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.641+0000 7fd3d52d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d0151be0 con 0x7fd3d00ae200 2023-12-12T09:54:18.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.641+0000 7fd3cffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d0150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44034/0 (socket says 172.21.15.72:44034) 2023-12-12T09:54:18.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cffff640 1 -- 172.21.15.72:0/900564555 learned_addr learned my addr 172.21.15.72:0/900564555 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3d4ad6640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d015e5d0 0x7fd3d00b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cffff640 1 -- 172.21.15.72:0/900564555 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d00ae200 msgr2=0x7fd3d01526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cffff640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d00ae200 0x7fd3d01526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cffff640 1 -- 172.21.15.72:0/900564555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d015e5d0 msgr2=0x7fd3d00b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cffff640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d015e5d0 0x7fd3d00b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cffff640 1 -- 172.21.15.72:0/900564555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c8009400 con 0x7fd3d0071260 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cffff640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d0150180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0007e40 tx=0x7fd3c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.642+0000 7fd3cd7fa640 1 -- 172.21.15.72:0/900564555 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0014620 con 0x7fd3d0071260 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.643+0000 7fd3cd7fa640 1 -- 172.21.15.72:0/900564555 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c000ee90 con 0x7fd3d0071260 2023-12-12T09:54:18.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.643+0000 7fd3cd7fa640 1 -- 172.21.15.72:0/900564555 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00149c0 con 0x7fd3d0071260 2023-12-12T09:54:18.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.643+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d0151ec0 con 0x7fd3d0071260 2023-12-12T09:54:18.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.643+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d00b2ab0 con 0x7fd3d0071260 2023-12-12T09:54:18.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.644+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d0074450 con 0x7fd3d0071260 2023-12-12T09:54:18.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.644+0000 7fd3cd7fa640 1 -- 172.21.15.72:0/900564555 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3c0014b20 con 0x7fd3d0071260 2023-12-12T09:54:18.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.646+0000 7fd3cd7fa640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3b006ae40 0x7fd3b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:18.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.646+0000 7fd3cd7fa640 1 -- 172.21.15.72:0/900564555 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd3c008a190 con 0x7fd3d0071260 2023-12-12T09:54:18.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.646+0000 7fd3c77fe640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3b006ae40 0x7fd3b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:18.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.647+0000 7fd3c77fe640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3b006ae40 0x7fd3b006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8007fd0 tx=0x7fd3b8007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:18.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.651+0000 7fd3cd7fa640 1 -- 172.21.15.72:0/900564555 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3c00550d0 con 0x7fd3d0071260 2023-12-12T09:54:18.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.882+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3d00adfe0 con 0x7fd3d0071260 2023-12-12T09:54:18.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.883+0000 7fd3cd7fa640 1 -- 172.21.15.72:0/900564555 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fd3c00586f0 con 0x7fd3d0071260 2023-12-12T09:54:18.885 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:18.886 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:18.886 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:18.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3b006ae40 msgr2=0x7fd3b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3b006ae40 0x7fd3b006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8007fd0 tx=0x7fd3b8007920 comp rx=0 tx=0).stop 2023-12-12T09:54:18.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 msgr2=0x7fd3d0150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:18.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d0150180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0007e40 tx=0x7fd3c000e4b0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 shutdown_connections 2023-12-12T09:54:18.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd3b006ae40 0x7fd3b006d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3d015e5d0 0x7fd3d00b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3d00ae200 0x7fd3d01526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 --2- 172.21.15.72:0/900564555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d0071260 0x7fd3d0150180 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:18.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 >> 172.21.15.72:0/900564555 conn(0x7fd3d00b0860 msgr2=0x7fd3d006cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:18.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 shutdown_connections 2023-12-12T09:54:18.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:18.886+0000 7fd3d52d7640 1 -- 172.21.15.72:0/900564555 wait complete. 2023-12-12T09:54:18.907 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:19.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.120+0000 7f12a5363640 1 Processor -- start 2023-12-12T09:54:19.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.120+0000 7f12a5363640 1 -- start start 2023-12-12T09:54:19.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.121+0000 7f12a5363640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00ae200 0x7f12a0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.121+0000 7f12a5363640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f12a00abd20 0x7f12a0071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.121+0000 7f12a5363640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a0071f90 con 0x7f12a015e5d0 2023-12-12T09:54:19.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.122+0000 7f12a5363640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a0072270 con 0x7f12a00abd20 2023-12-12T09:54:19.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.122+0000 7f12a5363640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0070070 con 0x7f12a00ae200 2023-12-12T09:54:19.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.122+0000 7f129ffff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f12a015e5d0 0x7f12a006bdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47334/0 (socket says 172.21.15.72:47334) 2023-12-12T09:54:19.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.122+0000 7f129ffff640 1 -- 172.21.15.72:0/2605397313 learned_addr learned my addr 172.21.15.72:0/2605397313 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.122+0000 7f12a4b62640 1 --2- 172.21.15.72:0/2605397313 >> v2:172.21.15.113:3300/0 conn(0x7f12a00abd20 0x7f12a0071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.122+0000 7f129f7fe640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00ae200 0x7f12a0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.122+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364271674 0 0) 0x7f12a0071f90 con 0x7f12a015e5d0 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.122+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1288003520 con 0x7f12a015e5d0 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621636712 0 0) 0x7f1288003520 con 0x7f12a015e5d0 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00ae200 msgr2=0x7f12a0070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00ae200 0x7f12a0070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 >> v2:172.21.15.113:3300/0 conn(0x7f12a00abd20 msgr2=0x7f12a0071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:19.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 --2- 172.21.15.72:0/2605397313 >> v2:172.21.15.113:3300/0 conn(0x7f12a00abd20 0x7f12a0071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a0070350 con 0x7f12a015e5d0 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f12900019f0 con 0x7f12a015e5d0 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 >> v1:172.21.15.72:6789/0 conn(0x7f12a015e5d0 legacy=0x7f12a006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1288003970 0x7f1288003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12880047c0 0x7f1288004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00ae200 0x7f1288005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0072270 con 0x7f1288003970 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0070070 con 0x7f12880047c0 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128800bab0 con 0x7f12a00ae200 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.123+0000 7f129ffff640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1288003970 0x7f1288003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.124+0000 7f12a4b62640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00ae200 0x7f1288005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.124+0000 7f129f7fe640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12880047c0 0x7f1288004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.124+0000 7f129ffff640 1 -- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00ae200 msgr2=0x7f1288005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.124+0000 7f129ffff640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00ae200 0x7f1288005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.124+0000 7f129ffff640 1 -- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12880047c0 msgr2=0x7f1288004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:19.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.124+0000 7f129ffff640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12880047c0 0x7f1288004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.124+0000 7f129ffff640 1 -- 172.21.15.72:0/2605397313 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1288003520 con 0x7f1288003970 2023-12-12T09:54:19.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.125+0000 7f129ffff640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1288003970 0x7f1288003d90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f129000a760 tx=0x7f12880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960165f3139536a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:19.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.125+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290034630 con 0x7f1288003970 2023-12-12T09:54:19.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.125+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129002f070 con 0x7f1288003970 2023-12-12T09:54:19.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.125+0000 7f129effd640 1 -- 172.21.15.72:0/2605397313 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290034990 con 0x7f1288003970 2023-12-12T09:54:19.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.126+0000 7f12a5363640 1 -- 172.21.15.72:0/2605397313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1288003970 msgr2=0x7f1288003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.126+0000 7f12a5363640 1 --2- 172.21.15.72:0/2605397313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1288003970 0x7f1288003d90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f129000a760 tx=0x7f12880057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.126+0000 7f12a4b62640 1 -- 172.21.15.72:0/2605397313 reap_dead start 2023-12-12T09:54:19.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.126+0000 7f12a5363640 1 -- 172.21.15.72:0/2605397313 shutdown_connections 2023-12-12T09:54:19.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.126+0000 7f12a5363640 1 -- 172.21.15.72:0/2605397313 >> 172.21.15.72:0/2605397313 conn(0x7f12a00b0860 msgr2=0x7f12a00b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:19.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.126+0000 7f12a5363640 1 -- 172.21.15.72:0/2605397313 shutdown_connections 2023-12-12T09:54:19.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.126+0000 7f12a5363640 1 -- 172.21.15.72:0/2605397313 wait complete. 2023-12-12T09:54:19.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.128+0000 7f12a5363640 1 Processor -- start 2023-12-12T09:54:19.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.128+0000 7f12a5363640 1 -- start start 2023-12-12T09:54:19.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a5363640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1288003970 0x7f12a00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a5363640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00cb7b0 0x7f12a01593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a5363640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12a00c7ee0 0x7f12a00c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a5363640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a015c5e0 con 0x7f12a00c7ee0 2023-12-12T09:54:19.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a5363640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00c98b0 con 0x7f1288003970 2023-12-12T09:54:19.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a5363640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00c88a0 con 0x7f12a00cb7b0 2023-12-12T09:54:19.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a4b62640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12a00c7ee0 0x7f12a00c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a4b62640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12a00c7ee0 0x7f12a00c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44062/0 (socket says 172.21.15.72:44062) 2023-12-12T09:54:19.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a4b62640 1 -- 172.21.15.72:0/2390186975 learned_addr learned my addr 172.21.15.72:0/2390186975 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:19.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f129ffff640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1288003970 0x7f12a00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a4b62640 1 -- 172.21.15.72:0/2390186975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00cb7b0 msgr2=0x7f12a01593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f129f7fe640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00cb7b0 0x7f12a01593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a4b62640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00cb7b0 0x7f12a01593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a4b62640 1 -- 172.21.15.72:0/2390186975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1288003970 msgr2=0x7f12a00c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a4b62640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1288003970 0x7f12a00c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.129+0000 7f12a4b62640 1 -- 172.21.15.72:0/2390186975 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a006c680 con 0x7f12a00c7ee0 2023-12-12T09:54:19.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.130+0000 7f12a4b62640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12a00c7ee0 0x7f12a00c8360 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f12a00cdc90 tx=0x7f1298010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:19.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.130+0000 7f129cff9640 1 -- 172.21.15.72:0/2390186975 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800e8a0 con 0x7f12a00c7ee0 2023-12-12T09:54:19.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.130+0000 7f129cff9640 1 -- 172.21.15.72:0/2390186975 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129800ea20 con 0x7f12a00c7ee0 2023-12-12T09:54:19.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.130+0000 7f129cff9640 1 -- 172.21.15.72:0/2390186975 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800edd0 con 0x7f12a00c7ee0 2023-12-12T09:54:19.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.130+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a00cbf00 con 0x7f12a00c7ee0 2023-12-12T09:54:19.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.130+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a015c7c0 con 0x7f12a00c7ee0 2023-12-12T09:54:19.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.131+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a00cc420 con 0x7f12a00c7ee0 2023-12-12T09:54:19.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.132+0000 7f129cff9640 1 -- 172.21.15.72:0/2390186975 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f12980143f0 con 0x7f12a00c7ee0 2023-12-12T09:54:19.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.134+0000 7f129cff9640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f128c06ae40 0x7f128c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.134+0000 7f129cff9640 1 -- 172.21.15.72:0/2390186975 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1298089540 con 0x7f12a00c7ee0 2023-12-12T09:54:19.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.134+0000 7f129ffff640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f128c06ae40 0x7f128c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.135+0000 7f129ffff640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f128c06ae40 0x7f128c06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1290034000 tx=0x7f1290009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:19.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.138+0000 7f129cff9640 1 -- 172.21.15.72:0/2390186975 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12a00cc420 con 0x7f12a00c7ee0 2023-12-12T09:54:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.367+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12a00cc420 con 0x7f12a00c7ee0 2023-12-12T09:54:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.368+0000 7f129cff9640 1 -- 172.21.15.72:0/2390186975 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f12a00cc420 con 0x7f12a00c7ee0 2023-12-12T09:54:19.370 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:19.370 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:53:47.254534+0000","modified":"2023-12-12T09:53:50.330369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.113:6835/211839795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":211839795},{"type":"v1","addr":"172.21.15.113:6835","nonce":211839795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-12T09:54:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 16 2023-12-12T09:54:19.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f128c06ae40 msgr2=0x7f128c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f128c06ae40 0x7f128c06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1290034000 tx=0x7f1290009030 comp rx=0 tx=0).stop 2023-12-12T09:54:19.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12a00c7ee0 msgr2=0x7f12a00c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12a00c7ee0 0x7f12a00c8360 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f12a00cdc90 tx=0x7f1298010c20 comp rx=0 tx=0).stop 2023-12-12T09:54:19.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 shutdown_connections 2023-12-12T09:54:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f128c06ae40 0x7f128c06d300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12a00c7ee0 0x7f12a00c8360 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12a00cb7b0 0x7f12a01593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.370+0000 7f12a5363640 1 --2- 172.21.15.72:0/2390186975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1288003970 0x7f12a00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.371+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 >> 172.21.15.72:0/2390186975 conn(0x7f12a00b0860 msgr2=0x7f12a006cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.371+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 shutdown_connections 2023-12-12T09:54:19.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.371+0000 7f12a5363640 1 -- 172.21.15.72:0/2390186975 wait complete. 2023-12-12T09:54:19.390 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:19.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.596+0000 7fe502248640 1 Processor -- start 2023-12-12T09:54:19.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.596+0000 7fe502248640 1 -- start start 2023-12-12T09:54:19.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.598+0000 7fe502248640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc072180 0x7fe4fc0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.598+0000 7fe502248640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0b3bb0 con 0x7fe4fc143e00 2023-12-12T09:54:19.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.598+0000 7fe502248640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc1441e0 con 0x7fe4fc06c410 2023-12-12T09:54:19.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.598+0000 7fe502248640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc072680 con 0x7fe4fc072180 2023-12-12T09:54:19.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.598+0000 7fe4fb7fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fe4fc143e00 0x7fe4fc0b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47344/0 (socket says 172.21.15.72:47344) 2023-12-12T09:54:19.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.598+0000 7fe4fb7fe640 1 -- 172.21.15.72:0/1817770713 learned_addr learned my addr 172.21.15.72:0/1817770713 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:19.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.598+0000 7fe4f3fff640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc072180 0x7fe4fc0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.598+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727066343 0 0) 0x7fe4fc0b3bb0 con 0x7fe4fc143e00 2023-12-12T09:54:19.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4e8003520 con 0x7fe4fc143e00 2023-12-12T09:54:19.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656395240 0 0) 0x7fe4e8003520 con 0x7fe4fc143e00 2023-12-12T09:54:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc072180 msgr2=0x7fe4fc0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc072180 0x7fe4fc0b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 >> v1:172.21.15.113:6789/0 conn(0x7fe4fc06c410 legacy=0x7fe4fc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc0b1e30 con 0x7fe4fc143e00 2023-12-12T09:54:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fe4ec001670 con 0x7fe4fc143e00 2023-12-12T09:54:19.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 >> v1:172.21.15.72:6789/0 conn(0x7fe4fc143e00 legacy=0x7fe4fc0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc072180 0x7fe4e80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4e8003f20 0x7fe4e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8004d00 0x7fe4e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc072680 con 0x7fe4e8003f20 2023-12-12T09:54:19.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e800ba60 con 0x7fe4e8004d00 2023-12-12T09:54:19.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e800bc20 con 0x7fe4fc072180 2023-12-12T09:54:19.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.599+0000 7fe4f3fff640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4e8003f20 0x7fe4e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.600+0000 7fe4f3fff640 1 -- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc072180 msgr2=0x7fe4e80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:19.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.600+0000 7fe4fb7fe640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc072180 0x7fe4e80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.600+0000 7fe4fbfff640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8004d00 0x7fe4e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.600+0000 7fe4f3fff640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc072180 0x7fe4e80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.600+0000 7fe4f3fff640 1 -- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8004d00 msgr2=0x7fe4e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:19.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.600+0000 7fe4f3fff640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8004d00 0x7fe4e8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.600+0000 7fe4f3fff640 1 -- 172.21.15.72:0/1817770713 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e8003520 con 0x7fe4e8003f20 2023-12-12T09:54:19.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.601+0000 7fe4f3fff640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4e8003f20 0x7fe4e80047c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009ac0 tx=0x7fe4e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514ff332ea90bfbf server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:19.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.601+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e402c6b0 con 0x7fe4e8003f20 2023-12-12T09:54:19.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.601+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e402c810 con 0x7fe4e8003f20 2023-12-12T09:54:19.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.601+0000 7fe4faffd640 1 -- 172.21.15.72:0/1817770713 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e402cba0 con 0x7fe4e8003f20 2023-12-12T09:54:19.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.602+0000 7fe502248640 1 -- 172.21.15.72:0/1817770713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4e8003f20 msgr2=0x7fe4e80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.602+0000 7fe502248640 1 --2- 172.21.15.72:0/1817770713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4e8003f20 0x7fe4e80047c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009ac0 tx=0x7fe4e80057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.602+0000 7fe4fbfff640 1 -- 172.21.15.72:0/1817770713 reap_dead start 2023-12-12T09:54:19.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.602+0000 7fe502248640 1 -- 172.21.15.72:0/1817770713 shutdown_connections 2023-12-12T09:54:19.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.602+0000 7fe502248640 1 -- 172.21.15.72:0/1817770713 >> 172.21.15.72:0/1817770713 conn(0x7fe4fc0b0a90 msgr2=0x7fe4fc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:19.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.602+0000 7fe502248640 1 -- 172.21.15.72:0/1817770713 shutdown_connections 2023-12-12T09:54:19.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.602+0000 7fe502248640 1 -- 172.21.15.72:0/1817770713 wait complete. 2023-12-12T09:54:19.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.604+0000 7fe502248640 1 Processor -- start 2023-12-12T09:54:19.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.604+0000 7fe502248640 1 -- start start 2023-12-12T09:54:19.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.605+0000 7fe502248640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8003f20 0x7fe4fc137cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.605+0000 7fe502248640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4fc1f3a10 0x7fe4fc0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.605+0000 7fe502248640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc15aae0 0x7fe4fc142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.605+0000 7fe502248640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0ca5a0 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.605+0000 7fe502248640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1f3e90 con 0x7fe4e8003f20 2023-12-12T09:54:19.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.605+0000 7fe502248640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc13ffc0 con 0x7fe4fc15aae0 2023-12-12T09:54:19.618 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.605+0000 7fe4f3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4fc1f3a10 0x7fe4fc0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4f3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4fc1f3a10 0x7fe4fc0ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44084/0 (socket says 172.21.15.72:44084) 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4f3fff640 1 -- 172.21.15.72:0/1491611571 learned_addr learned my addr 172.21.15.72:0/1491611571 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4fb7fe640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8003f20 0x7fe4fc137cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4f3fff640 1 -- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc15aae0 msgr2=0x7fe4fc142390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4fbfff640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc15aae0 0x7fe4fc142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4f3fff640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc15aae0 0x7fe4fc142390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4f3fff640 1 -- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8003f20 msgr2=0x7fe4fc137cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4f3fff640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8003f20 0x7fe4fc137cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4f3fff640 1 -- 172.21.15.72:0/1491611571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc0ca790 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4fb7fe640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8003f20 0x7fe4fc137cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.606+0000 7fe4f3fff640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4fc1f3a10 0x7fe4fc0ca060 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009ac0 tx=0x7fe4e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:19.619 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.607+0000 7fe4f8ff9640 1 -- 172.21.15.72:0/1491611571 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4003810 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.607+0000 7fe4f8ff9640 1 -- 172.21.15.72:0/1491611571 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e4003970 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.607+0000 7fe4f8ff9640 1 -- 172.21.15.72:0/1491611571 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4003d00 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.607+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc0caa70 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.607+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc140560 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.608+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc1406f0 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.609+0000 7fe4f8ff9640 1 -- 172.21.15.72:0/1491611571 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4e4040450 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.610+0000 7fe4f8ff9640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4dc06ae70 0x7fe4dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.610+0000 7fe4f8ff9640 1 -- 172.21.15.72:0/1491611571 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe4e40ab7a0 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.610+0000 7fe4fb7fe640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4dc06ae70 0x7fe4dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:19.620 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.611+0000 7fe4fb7fe640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4dc06ae70 0x7fe4dc06d330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0017a0 tx=0x7fe4ec001470 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:19.621 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.615+0000 7fe4f8ff9640 1 -- 172.21.15.72:0/1491611571 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4e4076650 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.804+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4fc0745d0 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.810+0000 7fe4f8ff9640 1 -- 172.21.15.72:0/1491611571 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16823 (secure 0 0 0) 0x7fe4e4003e60 con 0x7fe4fc1f3a10 2023-12-12T09:54:19.813 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:19.813 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":40,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:53:56.442722+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:53:45.083566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-12T09:53:46.274070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4dc06ae70 msgr2=0x7fe4dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4dc06ae70 0x7fe4dc06d330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0017a0 tx=0x7fe4ec001470 comp rx=0 tx=0).stop 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4fc1f3a10 msgr2=0x7fe4fc0ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4fc1f3a10 0x7fe4fc0ca060 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009ac0 tx=0x7fe4e4004730 comp rx=0 tx=0).stop 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 shutdown_connections 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe4dc06ae70 0x7fe4dc06d330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc15aae0 0x7fe4fc142390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4fc1f3a10 0x7fe4fc0ca060 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 --2- 172.21.15.72:0/1491611571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e8003f20 0x7fe4fc137cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.813+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 >> 172.21.15.72:0/1491611571 conn(0x7fe4fc0b0a90 msgr2=0x7fe4fc13e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.814+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 shutdown_connections 2023-12-12T09:54:19.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:19.814+0000 7fe502248640 1 -- 172.21.15.72:0/1491611571 wait complete. 2023-12-12T09:54:19.838 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-12T09:54:20.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.043+0000 7f11e7061640 1 Processor -- start 2023-12-12T09:54:20.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.044+0000 7f11e7061640 1 -- start start 2023-12-12T09:54:20.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.045+0000 7f11e7061640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f11e015e2b0 0x7f11e0164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.045+0000 7f11e7061640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e006ca40 con 0x7f11e0157a60 2023-12-12T09:54:20.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.045+0000 7f11e7061640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e0071cd0 con 0x7f11e006c410 2023-12-12T09:54:20.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.045+0000 7f11e7061640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e0070f10 con 0x7f11e015e2b0 2023-12-12T09:54:20.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.046+0000 7f11e4dfd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f11e0157a60 0x7f11e006de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47360/0 (socket says 172.21.15.72:47360) 2023-12-12T09:54:20.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.046+0000 7f11e55fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f11e015e2b0 0x7f11e0164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.046+0000 7f11e4dfd640 1 -- 172.21.15.72:0/1067226205 learned_addr learned my addr 172.21.15.72:0/1067226205 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:20.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.046+0000 7f11e55fe640 1 -- 172.21.15.72:0/1067226205 >> v1:172.21.15.113:6789/0 conn(0x7f11e006c410 legacy=0x7f11e015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:20.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.046+0000 7f11e55fe640 1 -- 172.21.15.72:0/1067226205 >> v1:172.21.15.72:6789/0 conn(0x7f11e0157a60 legacy=0x7f11e006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.046+0000 7f11e55fe640 1 -- 172.21.15.72:0/1067226205 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00711f0 con 0x7f11e015e2b0 2023-12-12T09:54:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.047+0000 7f11e55fe640 1 --2- 172.21.15.72:0/1067226205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e015e2b0 0x7f11e0164d00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009d90 tx=0x7f11d802e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78e22ca2ae72c0b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.047+0000 7f11df7fe640 1 -- 172.21.15.72:0/1067226205 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d802ede0 con 0x7f11e015e2b0 2023-12-12T09:54:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11df7fe640 1 -- 172.21.15.72:0/1067226205 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d8036440 con 0x7f11e015e2b0 2023-12-12T09:54:20.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11df7fe640 1 -- 172.21.15.72:0/1067226205 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d80367d0 con 0x7f11e015e2b0 2023-12-12T09:54:20.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11e7061640 1 -- 172.21.15.72:0/1067226205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e015e2b0 msgr2=0x7f11e0164d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11e7061640 1 --2- 172.21.15.72:0/1067226205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e015e2b0 0x7f11e0164d00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009d90 tx=0x7f11d802e030 comp rx=0 tx=0).stop 2023-12-12T09:54:20.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11e7061640 1 -- 172.21.15.72:0/1067226205 shutdown_connections 2023-12-12T09:54:20.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11e7061640 1 --2- 172.21.15.72:0/1067226205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e015e2b0 0x7f11e0164d00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11e7061640 1 -- 172.21.15.72:0/1067226205 >> 172.21.15.72:0/1067226205 conn(0x7f11e00b0a90 msgr2=0x7f11e00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:20.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11e7061640 1 -- 172.21.15.72:0/1067226205 shutdown_connections 2023-12-12T09:54:20.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.048+0000 7f11e7061640 1 -- 172.21.15.72:0/1067226205 wait complete. 2023-12-12T09:54:20.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.050+0000 7f11e7061640 1 Processor -- start 2023-12-12T09:54:20.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.050+0000 7f11e7061640 1 -- start start 2023-12-12T09:54:20.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.051+0000 7f11e7061640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e006c410 0x7f11e0153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.051+0000 7f11e7061640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e0157a60 0x7f11e0158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.051+0000 7f11e7061640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11e015e2b0 0x7f11e0159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.051+0000 7f11e7061640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e015d330 con 0x7f11e0157a60 2023-12-12T09:54:20.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.051+0000 7f11e7061640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0153e60 con 0x7f11e006c410 2023-12-12T09:54:20.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.051+0000 7f11e7061640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e015c6b0 con 0x7f11e015e2b0 2023-12-12T09:54:20.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.051+0000 7f11dffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e0157a60 0x7f11e0158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.051+0000 7f11dffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e0157a60 0x7f11e0158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44112/0 (socket says 172.21.15.72:44112) 2023-12-12T09:54:20.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11dffff640 1 -- 172.21.15.72:0/3711715601 learned_addr learned my addr 172.21.15.72:0/3711715601 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:20.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11e4dfd640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e006c410 0x7f11e0153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11e55fe640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11e015e2b0 0x7f11e0159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11dffff640 1 -- 172.21.15.72:0/3711715601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11e015e2b0 msgr2=0x7f11e0159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11dffff640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11e015e2b0 0x7f11e0159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11dffff640 1 -- 172.21.15.72:0/3711715601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e006c410 msgr2=0x7f11e0153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11dffff640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e006c410 0x7f11e0153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11dffff640 1 -- 172.21.15.72:0/3711715601 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d8009a70 con 0x7f11e0157a60 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.052+0000 7f11dffff640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e0157a60 0x7f11e0158bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c430 tx=0x7f11cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.053+0000 7f11dd7fa640 1 -- 172.21.15.72:0/3711715601 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc011790 con 0x7f11e0157a60 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.053+0000 7f11dd7fa640 1 -- 172.21.15.72:0/3711715601 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc0118f0 con 0x7f11e0157a60 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.053+0000 7f11dd7fa640 1 -- 172.21.15.72:0/3711715601 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc011ca0 con 0x7f11e0157a60 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.053+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e015c990 con 0x7f11e0157a60 2023-12-12T09:54:20.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.053+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e015cf50 con 0x7f11e0157a60 2023-12-12T09:54:20.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.054+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e015d470 con 0x7f11e0157a60 2023-12-12T09:54:20.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.055+0000 7f11dd7fa640 1 -- 172.21.15.72:0/3711715601 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11cc01c460 con 0x7f11e0157a60 2023-12-12T09:54:20.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.056+0000 7f11dd7fa640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11c406ae40 0x7f11c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.056+0000 7f11dd7fa640 1 -- 172.21.15.72:0/3711715601 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f11cc089690 con 0x7f11e0157a60 2023-12-12T09:54:20.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.056+0000 7f11e4dfd640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11c406ae40 0x7f11c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.057+0000 7f11e4dfd640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11c406ae40 0x7f11c406d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005880 tx=0x7f11d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:20.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.062+0000 7f11dd7fa640 1 -- 172.21.15.72:0/3711715601 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11e015d470 con 0x7f11e0157a60 2023-12-12T09:54:20.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.292+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f11e015d470 con 0x7f11e0157a60 2023-12-12T09:54:20.295 INFO:tasks.ceph.mon.a.smithi072.stderr:2023-12-12T09:54:20.293+0000 7f3fa4ad2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-12T09:54:20.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.297+0000 7f11dd7fa640 1 -- 172.21.15.72:0/3711715601 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f11e015d470 con 0x7f11e0157a60 2023-12-12T09:54:20.299 INFO:teuthology.orchestra.run.smithi072.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-12T09:54:20.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.300+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11c406ae40 msgr2=0x7f11c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.300+0000 7f11e7061640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11c406ae40 0x7f11c406d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005880 tx=0x7f11d4009040 comp rx=0 tx=0).stop 2023-12-12T09:54:20.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e0157a60 msgr2=0x7f11e0158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e0157a60 0x7f11e0158bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c430 tx=0x7f11cc00c8f0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 shutdown_connections 2023-12-12T09:54:20.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11c406ae40 0x7f11c406d300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11e015e2b0 0x7f11e0159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e0157a60 0x7f11e0158bf0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 --2- 172.21.15.72:0/3711715601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11e006c410 0x7f11e0153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 >> 172.21.15.72:0/3711715601 conn(0x7f11e00b0a90 msgr2=0x7f11e006dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:20.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 shutdown_connections 2023-12-12T09:54:20.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.301+0000 7f11e7061640 1 -- 172.21.15.72:0/3711715601 wait complete. 2023-12-12T09:54:20.322 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-12T09:54:20.419 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-12-12T09:54:20.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.540+0000 7f3dc7fff640 1 Processor -- start 2023-12-12T09:54:20.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.540+0000 7f3dc7fff640 1 -- start start 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.541+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc8155400 0x7f3dc806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc7fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3dc80ce150 0x7f3dc80ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3dc80cedf0 0x7f3dc80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc7fff640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc80cf7c0 con 0x7f3dc80ce150 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc7fff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc815b4a0 con 0x7f3dc80cedf0 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc7fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80adbb0 con 0x7f3dc8155400 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc67fc640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3dc80ce150 0x7f3dc80ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc67fc640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3dc80ce150 0x7f3dc80ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44124/0 (socket says 172.21.15.72:44124) 2023-12-12T09:54:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc67fc640 1 -- 172.21.15.72:0/3692114342 learned_addr learned my addr 172.21.15.72:0/3692114342 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc67fc640 1 -- 172.21.15.72:0/3692114342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc8155400 msgr2=0x7f3dc806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc6ffd640 1 --2- 172.21.15.72:0/3692114342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc8155400 0x7f3dc806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc77fe640 1 --2- 172.21.15.72:0/3692114342 >> v2:172.21.15.113:3300/0 conn(0x7f3dc80cedf0 0x7f3dc80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc67fc640 1 --2- 172.21.15.72:0/3692114342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc8155400 0x7f3dc806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc67fc640 1 -- 172.21.15.72:0/3692114342 >> v2:172.21.15.113:3300/0 conn(0x7f3dc80cedf0 msgr2=0x7f3dc80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc67fc640 1 --2- 172.21.15.72:0/3692114342 >> v2:172.21.15.113:3300/0 conn(0x7f3dc80cedf0 0x7f3dc80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.542+0000 7f3dc67fc640 1 -- 172.21.15.72:0/3692114342 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc80ade90 con 0x7f3dc80ce150 2023-12-12T09:54:20.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.543+0000 7f3dc77fe640 1 --2- 172.21.15.72:0/3692114342 >> v2:172.21.15.113:3300/0 conn(0x7f3dc80cedf0 0x7f3dc80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:20.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.544+0000 7f3dc67fc640 1 --2- 172.21.15.72:0/3692114342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc80ce150 0x7f3dc80ce5b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3dc806cf30 tx=0x7f3dbc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7625041217ca24a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.544+0000 7f3dc5ffb640 1 -- 172.21.15.72:0/3692114342 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc031860 con 0x7f3dc80ce150 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.544+0000 7f3dc5ffb640 1 -- 172.21.15.72:0/3692114342 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc0319c0 con 0x7f3dc80ce150 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.544+0000 7f3dc5ffb640 1 -- 172.21.15.72:0/3692114342 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc031d50 con 0x7f3dc80ce150 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.544+0000 7f3dc7fff640 1 -- 172.21.15.72:0/3692114342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc80ce150 msgr2=0x7f3dc80ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.544+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/3692114342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc80ce150 0x7f3dc80ce5b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3dc806cf30 tx=0x7f3dbc02b900 comp rx=0 tx=0).stop 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.545+0000 7f3dc7fff640 1 -- 172.21.15.72:0/3692114342 shutdown_connections 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.545+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/3692114342 >> v2:172.21.15.113:3300/0 conn(0x7f3dc80cedf0 0x7f3dc80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.545+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/3692114342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc80ce150 0x7f3dc80ce5b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.545+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/3692114342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc8155400 0x7f3dc806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.545+0000 7f3dc7fff640 1 -- 172.21.15.72:0/3692114342 >> 172.21.15.72:0/3692114342 conn(0x7f3dc806a640 msgr2=0x7f3dc806aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.545+0000 7f3dc7fff640 1 -- 172.21.15.72:0/3692114342 shutdown_connections 2023-12-12T09:54:20.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.545+0000 7f3dc7fff640 1 -- 172.21.15.72:0/3692114342 wait complete. 2023-12-12T09:54:20.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.546+0000 7f3dc7fff640 1 Processor -- start 2023-12-12T09:54:20.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.547+0000 7f3dc7fff640 1 -- start start 2023-12-12T09:54:20.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc80ce150 0x7f3dc80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc80cedf0 0x7f3dc809c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc8155400 0x7f3dc80b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc7fff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc8140c90 con 0x7f3dc8155400 2023-12-12T09:54:20.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc7fff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc809c980 con 0x7f3dc80ce150 2023-12-12T09:54:20.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc7fff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc8099b90 con 0x7f3dc80cedf0 2023-12-12T09:54:20.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc8155400 0x7f3dc80b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc8155400 0x7f3dc80b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44134/0 (socket says 172.21.15.72:44134) 2023-12-12T09:54:20.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc77fe640 1 -- 172.21.15.72:0/4129547765 learned_addr learned my addr 172.21.15.72:0/4129547765 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:20.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc67fc640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc80cedf0 0x7f3dc809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc6ffd640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc80ce150 0x7f3dc80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc77fe640 1 -- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc80cedf0 msgr2=0x7f3dc809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc77fe640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc80cedf0 0x7f3dc809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc77fe640 1 -- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc80ce150 msgr2=0x7f3dc80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc77fe640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc80ce150 0x7f3dc80ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.548+0000 7f3dc77fe640 1 -- 172.21.15.72:0/4129547765 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc0094d0 con 0x7f3dc8155400 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.549+0000 7f3dc6ffd640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc80ce150 0x7f3dc80ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.549+0000 7f3dc67fc640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc80cedf0 0x7f3dc809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.549+0000 7f3dc77fe640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc8155400 0x7f3dc80b0680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000cef0 tx=0x7f3dc000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.549+0000 7f3daf7fe640 1 -- 172.21.15.72:0/4129547765 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000d840 con 0x7f3dc8155400 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.549+0000 7f3daf7fe640 1 -- 172.21.15.72:0/4129547765 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc000d9c0 con 0x7f3dc8155400 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.549+0000 7f3daf7fe640 1 -- 172.21.15.72:0/4129547765 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000dd70 con 0x7f3dc8155400 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.549+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc8099e70 con 0x7f3dc8155400 2023-12-12T09:54:20.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.549+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc809a430 con 0x7f3dc8155400 2023-12-12T09:54:20.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.550+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc809a650 con 0x7f3dc8155400 2023-12-12T09:54:20.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.551+0000 7f3daf7fe640 1 -- 172.21.15.72:0/4129547765 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3dc001e3b0 con 0x7f3dc8155400 2023-12-12T09:54:20.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.552+0000 7f3daf7fe640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3db406ae70 0x7f3db406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:20.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.552+0000 7f3daf7fe640 1 -- 172.21.15.72:0/4129547765 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f3dc008a210 con 0x7f3dc8155400 2023-12-12T09:54:20.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.552+0000 7f3dc6ffd640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3db406ae70 0x7f3db406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:20.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.553+0000 7f3dc6ffd640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3db406ae70 0x7f3db406d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3db800b800 tx=0x7f3db8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:20.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.556+0000 7f3daf7fe640 1 -- 172.21.15.72:0/4129547765 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3dc0055220 con 0x7f3dc8155400 2023-12-12T09:54:20.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:20.789+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3dc8074450 con 0x7f3dc8155400 2023-12-12T09:54:21.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.300+0000 7f3daf7fe640 1 -- 172.21.15.72:0/4129547765 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3dc0058840 con 0x7f3dc8155400 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3db406ae70 msgr2=0x7f3db406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3db406ae70 0x7f3db406d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3db800b800 tx=0x7f3db8008040 comp rx=0 tx=0).stop 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc8155400 msgr2=0x7f3dc80b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc8155400 0x7f3dc80b0680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000cef0 tx=0x7f3dc000a710 comp rx=0 tx=0).stop 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 shutdown_connections 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3db406ae70 0x7f3db406d330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dc8155400 0x7f3dc80b0680 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc80cedf0 0x7f3dc809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 --2- 172.21.15.72:0/4129547765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc80ce150 0x7f3dc80ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 >> 172.21.15.72:0/4129547765 conn(0x7f3dc806a640 msgr2=0x7f3dc80b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:21.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 shutdown_connections 2023-12-12T09:54:21.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.305+0000 7f3dc7fff640 1 -- 172.21.15.72:0/4129547765 wait complete. 2023-12-12T09:54:21.328 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-12T09:54:21.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.546+0000 7fe1c2644640 1 Processor -- start 2023-12-12T09:54:21.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.547+0000 7fe1c2644640 1 -- start start 2023-12-12T09:54:21.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.548+0000 7fe1c2644640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc0d15b0 0x7fe1bc0d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:21.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.548+0000 7fe1c2644640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1bc0d1990 0x7fe1bc0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:21.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.548+0000 7fe1c2644640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc0ce5d0 con 0x7fe1bc15cd90 2023-12-12T09:54:21.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.548+0000 7fe1c2644640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc15d8f0 con 0x7fe1bc0d1990 2023-12-12T09:54:21.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.548+0000 7fe1c2644640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0ce870 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.549+0000 7fe1c1642640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc0d15b0 0x7fe1bc0d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.549+0000 7fe1c1642640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc0d15b0 0x7fe1bc0d3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:55600/0 (socket says 172.21.15.72:55600) 2023-12-12T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.549+0000 7fe1c1642640 1 -- 172.21.15.72:0/698820882 learned_addr learned my addr 172.21.15.72:0/698820882 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.549+0000 7fe1c0e41640 1 --2- 172.21.15.72:0/698820882 >> v2:172.21.15.113:3300/0 conn(0x7fe1bc0d1990 0x7fe1bc0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.549+0000 7fe1c1642640 1 -- 172.21.15.72:0/698820882 >> v2:172.21.15.113:3300/0 conn(0x7fe1bc0d1990 msgr2=0x7fe1bc0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.549+0000 7fe1c1642640 1 --2- 172.21.15.72:0/698820882 >> v2:172.21.15.113:3300/0 conn(0x7fe1bc0d1990 0x7fe1bc0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.549+0000 7fe1c1642640 1 -- 172.21.15.72:0/698820882 >> v1:172.21.15.113:6789/0 conn(0x7fe1bc15cd90 legacy=0x7fe1bc15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:21.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.549+0000 7fe1c1642640 1 -- 172.21.15.72:0/698820882 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0ceb50 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.551+0000 7fe1c1642640 1 --2- 172.21.15.72:0/698820882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc0d15b0 0x7fe1bc0d3a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009e50 tx=0x7fe1ac02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1424b9889dc9cc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:21.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.551+0000 7fe1bbfff640 1 -- 172.21.15.72:0/698820882 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac02ca80 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.551+0000 7fe1bbfff640 1 -- 172.21.15.72:0/698820882 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac02cbe0 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.551+0000 7fe1bbfff640 1 -- 172.21.15.72:0/698820882 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0343f0 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.552+0000 7fe1c2644640 1 -- 172.21.15.72:0/698820882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc0d15b0 msgr2=0x7fe1bc0d3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:21.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.552+0000 7fe1c2644640 1 --2- 172.21.15.72:0/698820882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc0d15b0 0x7fe1bc0d3a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009e50 tx=0x7fe1ac02bee0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.552+0000 7fe1c2644640 1 -- 172.21.15.72:0/698820882 shutdown_connections 2023-12-12T09:54:21.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.552+0000 7fe1c2644640 1 --2- 172.21.15.72:0/698820882 >> v2:172.21.15.113:3300/0 conn(0x7fe1bc0d1990 0x7fe1bc0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.552+0000 7fe1c2644640 1 --2- 172.21.15.72:0/698820882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc0d15b0 0x7fe1bc0d3a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.552+0000 7fe1c2644640 1 -- 172.21.15.72:0/698820882 >> 172.21.15.72:0/698820882 conn(0x7fe1bc0d2f10 msgr2=0x7fe1bc0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:21.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.552+0000 7fe1c2644640 1 -- 172.21.15.72:0/698820882 shutdown_connections 2023-12-12T09:54:21.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.552+0000 7fe1c2644640 1 -- 172.21.15.72:0/698820882 wait complete. 2023-12-12T09:54:21.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.554+0000 7fe1c2644640 1 Processor -- start 2023-12-12T09:54:21.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.554+0000 7fe1c2644640 1 -- start start 2023-12-12T09:54:21.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c2644640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1bc0d15b0 0x7fe1bc0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:21.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c2644640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1bc0d1990 0x7fe1bc09eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c2644640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc15cd90 0x7fe1bc09f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c2644640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0cab70 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c2644640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0cae50 con 0x7fe1bc0d1990 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c2644640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0cb130 con 0x7fe1bc15cd90 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c1642640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1bc0d15b0 0x7fe1bc0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c1642640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1bc0d15b0 0x7fe1bc0ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44140/0 (socket says 172.21.15.72:44140) 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.555+0000 7fe1c1642640 1 -- 172.21.15.72:0/4031595842 learned_addr learned my addr 172.21.15.72:0/4031595842 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1c0e41640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1bc0d1990 0x7fe1bc09eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1c1642640 1 -- 172.21.15.72:0/4031595842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc15cd90 msgr2=0x7fe1bc09f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1c1642640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc15cd90 0x7fe1bc09f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1c1642640 1 -- 172.21.15.72:0/4031595842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1bc0d1990 msgr2=0x7fe1bc09eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1c1642640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1bc0d1990 0x7fe1bc09eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:21.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1c1642640 1 -- 172.21.15.72:0/4031595842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac009b30 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1c1642640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1bc0d15b0 0x7fe1bc0ccb90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac036040 tx=0x7fe1ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:21.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1b9ffb640 1 -- 172.21.15.72:0/4031595842 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac009510 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1b9ffb640 1 -- 172.21.15.72:0/4031595842 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac03e450 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc0cb410 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.556+0000 7fe1b9ffb640 1 -- 172.21.15.72:0/4031595842 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac03e820 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.557+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc0cd0d0 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.558+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc0cd260 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.559+0000 7fe1b9ffb640 1 -- 172.21.15.72:0/4031595842 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe1ac02cd50 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.559+0000 7fe1b9ffb640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe1a806ae40 0x7fe1a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:21.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.560+0000 7fe1b9ffb640 1 -- 172.21.15.72:0/4031595842 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fe1ac0ac1c0 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.560+0000 7fe1c0e41640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe1a806ae40 0x7fe1a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:21.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.560+0000 7fe1c0e41640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe1a806ae40 0x7fe1a806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0b8be0 tx=0x7fe1b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:21.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.564+0000 7fe1b9ffb640 1 -- 172.21.15.72:0/4031595842 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1ac077150 con 0x7fe1bc0d15b0 2023-12-12T09:54:21.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:21.755+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe1bc074450 con 0x7fe1bc0d15b0 2023-12-12T09:54:22.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.310+0000 7fe1b9ffb640 1 -- 172.21.15.72:0/4031595842 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe1ac07a770 con 0x7fe1bc0d15b0 2023-12-12T09:54:22.312 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' removed 2023-12-12T09:54:22.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.313+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe1a806ae40 msgr2=0x7fe1a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:22.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe1a806ae40 0x7fe1a806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0b8be0 tx=0x7fe1b0008040 comp rx=0 tx=0).stop 2023-12-12T09:54:22.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1bc0d15b0 msgr2=0x7fe1bc0ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:22.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1bc0d15b0 0x7fe1bc0ccb90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac036040 tx=0x7fe1ac004a40 comp rx=0 tx=0).stop 2023-12-12T09:54:22.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 shutdown_connections 2023-12-12T09:54:22.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe1a806ae40 0x7fe1a806d300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:22.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1bc15cd90 0x7fe1bc09f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:22.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1bc0d1990 0x7fe1bc09eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:22.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 --2- 172.21.15.72:0/4031595842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1bc0d15b0 0x7fe1bc0ccb90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:22.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 >> 172.21.15.72:0/4031595842 conn(0x7fe1bc0d2f10 msgr2=0x7fe1bc0b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:22.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 shutdown_connections 2023-12-12T09:54:22.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.314+0000 7fe1c2644640 1 -- 172.21.15.72:0/4031595842 wait complete. 2023-12-12T09:54:22.336 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-12T09:54:22.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.545+0000 7feeddd89640 1 Processor -- start 2023-12-12T09:54:22.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.545+0000 7feeddd89640 1 -- start start 2023-12-12T09:54:22.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.546+0000 7feeddd89640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7feed80ce370 0x7feed806ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:22.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feeddd89640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed80ce750 con 0x7feed806d200 2023-12-12T09:54:22.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feeddd89640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed8159e50 con 0x7feed80712d0 2023-12-12T09:54:22.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feeddd89640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed815a130 con 0x7feed80ce370 2023-12-12T09:54:22.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feed77fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7feed80ce370 0x7feed806ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:22.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feed6ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7feed806d200 0x7feed8070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47370/0 (socket says 172.21.15.72:47370) 2023-12-12T09:54:22.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feed6ffd640 1 -- 172.21.15.72:0/1317019117 learned_addr learned my addr 172.21.15.72:0/1317019117 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:22.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feed77fe640 1 -- 172.21.15.72:0/1317019117 >> v1:172.21.15.113:6789/0 conn(0x7feed80712d0 legacy=0x7feed80cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:22.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feed77fe640 1 -- 172.21.15.72:0/1317019117 >> v1:172.21.15.72:6789/0 conn(0x7feed806d200 legacy=0x7feed8070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:22.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.547+0000 7feed77fe640 1 -- 172.21.15.72:0/1317019117 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed815a410 con 0x7feed80ce370 2023-12-12T09:54:22.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.548+0000 7feed77fe640 1 --2- 172.21.15.72:0/1317019117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80ce370 0x7feed806ccc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feed8074140 tx=0x7feec802de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2735542a1f7c7da8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:22.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feed67fc640 1 -- 172.21.15.72:0/1317019117 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec802ebb0 con 0x7feed80ce370 2023-12-12T09:54:22.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feed67fc640 1 -- 172.21.15.72:0/1317019117 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec802ed10 con 0x7feed80ce370 2023-12-12T09:54:22.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feed67fc640 1 -- 172.21.15.72:0/1317019117 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8036660 con 0x7feed80ce370 2023-12-12T09:54:22.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feeddd89640 1 -- 172.21.15.72:0/1317019117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80ce370 msgr2=0x7feed806ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:22.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feeddd89640 1 --2- 172.21.15.72:0/1317019117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80ce370 0x7feed806ccc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feed8074140 tx=0x7feec802de00 comp rx=0 tx=0).stop 2023-12-12T09:54:22.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feeddd89640 1 -- 172.21.15.72:0/1317019117 shutdown_connections 2023-12-12T09:54:22.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feeddd89640 1 --2- 172.21.15.72:0/1317019117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80ce370 0x7feed806ccc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:22.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feeddd89640 1 -- 172.21.15.72:0/1317019117 >> 172.21.15.72:0/1317019117 conn(0x7feed80b0a90 msgr2=0x7feed80b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:22.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feeddd89640 1 -- 172.21.15.72:0/1317019117 shutdown_connections 2023-12-12T09:54:22.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.549+0000 7feeddd89640 1 -- 172.21.15.72:0/1317019117 wait complete. 2023-12-12T09:54:22.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.551+0000 7feeddd89640 1 Processor -- start 2023-12-12T09:54:22.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.551+0000 7feeddd89640 1 -- start start 2023-12-12T09:54:22.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.552+0000 7feeddd89640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed806d200 0x7feed81e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:22.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.552+0000 7feeddd89640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80712d0 0x7feed8151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:22.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.552+0000 7feeddd89640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed80ce370 0x7feed8142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:22.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.552+0000 7feeddd89640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8143660 con 0x7feed80712d0 2023-12-12T09:54:22.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.552+0000 7feeddd89640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8142d50 con 0x7feed80ce370 2023-12-12T09:54:22.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.552+0000 7feeddd89640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed81e4ac0 con 0x7feed806d200 2023-12-12T09:54:22.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.552+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80712d0 0x7feed8151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:22.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80712d0 0x7feed8151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44148/0 (socket says 172.21.15.72:44148) 2023-12-12T09:54:22.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed6ffd640 1 -- 172.21.15.72:0/3038547573 learned_addr learned my addr 172.21.15.72:0/3038547573 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed77fe640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed806d200 0x7feed81e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed7fff640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed80ce370 0x7feed8142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed6ffd640 1 -- 172.21.15.72:0/3038547573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed806d200 msgr2=0x7feed81e44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed6ffd640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed806d200 0x7feed81e44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed6ffd640 1 -- 172.21.15.72:0/3038547573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed80ce370 msgr2=0x7feed8142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed6ffd640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed80ce370 0x7feed8142810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed6ffd640 1 -- 172.21.15.72:0/3038547573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec800a5a0 con 0x7feed80712d0 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feed6ffd640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80712d0 0x7feed8151b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feec000c4a0 tx=0x7feec000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feecffff640 1 -- 172.21.15.72:0/3038547573 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0005ce0 con 0x7feed80712d0 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feecffff640 1 -- 172.21.15.72:0/3038547573 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0005e40 con 0x7feed80712d0 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.553+0000 7feecffff640 1 -- 172.21.15.72:0/3038547573 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0007270 con 0x7feed80712d0 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.554+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed81e4da0 con 0x7feed80712d0 2023-12-12T09:54:22.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.554+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed8143280 con 0x7feed80712d0 2023-12-12T09:54:22.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.555+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed81437a0 con 0x7feed80712d0 2023-12-12T09:54:22.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.555+0000 7feecffff640 1 -- 172.21.15.72:0/3038547573 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feec00073d0 con 0x7feed80712d0 2023-12-12T09:54:22.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.556+0000 7feecffff640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feeb806ae40 0x7feeb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:22.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.556+0000 7feecffff640 1 -- 172.21.15.72:0/3038547573 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5706+0+0 (secure 0 0 0) 0x7feec0088af0 con 0x7feed80712d0 2023-12-12T09:54:22.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.557+0000 7feed77fe640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feeb806ae40 0x7feeb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:22.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.557+0000 7feed77fe640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feeb806ae40 0x7feeb806d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feec8038000 tx=0x7feec8032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:22.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.561+0000 7feecffff640 1 -- 172.21.15.72:0/3038547573 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feec0053bb0 con 0x7feed80712d0 2023-12-12T09:54:22.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:22.738+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feed8074670 con 0x7feed80712d0 2023-12-12T09:54:23.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.321+0000 7feecffff640 1 -- 172.21.15.72:0/3038547573 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7feec0017020 con 0x7feed80712d0 2023-12-12T09:54:23.323 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' removed 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feeb806ae40 msgr2=0x7feeb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feeb806ae40 0x7feeb806d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feec8038000 tx=0x7feec8032000 comp rx=0 tx=0).stop 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80712d0 msgr2=0x7feed8151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80712d0 0x7feed8151b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feec000c4a0 tx=0x7feec000c960 comp rx=0 tx=0).stop 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 shutdown_connections 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feeb806ae40 0x7feeb806d300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed80ce370 0x7feed8142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feed80712d0 0x7feed8151b10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 --2- 172.21.15.72:0/3038547573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed806d200 0x7feed81e44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.325+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 >> 172.21.15.72:0/3038547573 conn(0x7feed80b0a90 msgr2=0x7feed80caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:23.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.326+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 shutdown_connections 2023-12-12T09:54:23.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.326+0000 7feeddd89640 1 -- 172.21.15.72:0/3038547573 wait complete. 2023-12-12T09:54:23.352 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-12T09:54:23.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.550+0000 7f077d275640 1 Processor -- start 2023-12-12T09:54:23.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.551+0000 7f077d275640 1 -- start start 2023-12-12T09:54:23.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.552+0000 7f077d275640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f077815cd90 0x7f077806a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:23.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.552+0000 7f077d275640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f077815d1f0 0x7f077806a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:23.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.552+0000 7f077d275640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077815dd30 con 0x7f0778159360 2023-12-12T09:54:23.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.552+0000 7f077d275640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077806d1b0 con 0x7f077815d1f0 2023-12-12T09:54:23.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.552+0000 7f077d275640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077815d770 con 0x7f077815cd90 2023-12-12T09:54:23.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.552+0000 7f07777fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f077815d1f0 0x7f077806a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:23.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.553+0000 7f07777fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f077815d1f0 0x7f077806a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44158/0 (socket says 172.21.15.72:44158) 2023-12-12T09:54:23.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.553+0000 7f07777fe640 1 -- 172.21.15.72:0/2611027191 learned_addr learned my addr 172.21.15.72:0/2611027191 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:23.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.553+0000 7f07777fe640 1 -- 172.21.15.72:0/2611027191 >> v2:172.21.15.113:3300/0 conn(0x7f077815cd90 msgr2=0x7f077806a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:23.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.553+0000 7f07767fc640 1 --2- 172.21.15.72:0/2611027191 >> v2:172.21.15.113:3300/0 conn(0x7f077815cd90 0x7f077806a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:23.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.553+0000 7f07777fe640 1 --2- 172.21.15.72:0/2611027191 >> v2:172.21.15.113:3300/0 conn(0x7f077815cd90 0x7f077806a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.553+0000 7f07777fe640 1 -- 172.21.15.72:0/2611027191 >> v1:172.21.15.113:6789/0 conn(0x7f0778159360 legacy=0x7f077806caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:23.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.553+0000 7f07777fe640 1 -- 172.21.15.72:0/2611027191 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077815da50 con 0x7f077815d1f0 2023-12-12T09:54:23.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.553+0000 7f07767fc640 1 --2- 172.21.15.72:0/2611027191 >> v2:172.21.15.113:3300/0 conn(0x7f077815cd90 0x7f077806a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:23.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.554+0000 7f07777fe640 1 --2- 172.21.15.72:0/2611027191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f077815d1f0 0x7f077806a9a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f07780b1830 tx=0x7f077002cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1599b5d3e67c2eb server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:23.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.554+0000 7f0775ffb640 1 -- 172.21.15.72:0/2611027191 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077002d900 con 0x7f077815d1f0 2023-12-12T09:54:23.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.554+0000 7f0775ffb640 1 -- 172.21.15.72:0/2611027191 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077002da60 con 0x7f077815d1f0 2023-12-12T09:54:23.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.554+0000 7f0775ffb640 1 -- 172.21.15.72:0/2611027191 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077002ddf0 con 0x7f077815d1f0 2023-12-12T09:54:23.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.555+0000 7f077d275640 1 -- 172.21.15.72:0/2611027191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f077815d1f0 msgr2=0x7f077806a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:23.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.555+0000 7f077d275640 1 --2- 172.21.15.72:0/2611027191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f077815d1f0 0x7f077806a9a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f07780b1830 tx=0x7f077002cf60 comp rx=0 tx=0).stop 2023-12-12T09:54:23.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.555+0000 7f077d275640 1 -- 172.21.15.72:0/2611027191 shutdown_connections 2023-12-12T09:54:23.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.555+0000 7f077d275640 1 --2- 172.21.15.72:0/2611027191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f077815d1f0 0x7f077806a9a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.555+0000 7f077d275640 1 --2- 172.21.15.72:0/2611027191 >> v2:172.21.15.113:3300/0 conn(0x7f077815cd90 0x7f077806a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.555+0000 7f077d275640 1 -- 172.21.15.72:0/2611027191 >> 172.21.15.72:0/2611027191 conn(0x7f07780cee70 msgr2=0x7f07780cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:23.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.555+0000 7f077d275640 1 -- 172.21.15.72:0/2611027191 shutdown_connections 2023-12-12T09:54:23.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.555+0000 7f077d275640 1 -- 172.21.15.72:0/2611027191 wait complete. 2023-12-12T09:54:23.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.557+0000 7f077d275640 1 Processor -- start 2023-12-12T09:54:23.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.557+0000 7f077d275640 1 -- start start 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.558+0000 7f077d275640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0778159360 0x7f077809be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.558+0000 7f077d275640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077815cd90 0x7f07780c1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.558+0000 7f077d275640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077815d1f0 0x7f07780cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.558+0000 7f077d275640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077809c3d0 con 0x7f0778159360 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.558+0000 7f077d275640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07781584e0 con 0x7f077815cd90 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.558+0000 7f077d275640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0778157b00 con 0x7f077815d1f0 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0778159360 0x7f077809be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0778159360 0x7f077809be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44166/0 (socket says 172.21.15.72:44166) 2023-12-12T09:54:23.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 -- 172.21.15.72:0/785158136 learned_addr learned my addr 172.21.15.72:0/785158136 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:23.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f07767fc640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077815cd90 0x7f07780c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:23.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 -- 172.21.15.72:0/785158136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077815d1f0 msgr2=0x7f07780cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:23.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077815d1f0 0x7f07780cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 -- 172.21.15.72:0/785158136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077815cd90 msgr2=0x7f07780c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:23.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077815cd90 0x7f07780c1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 -- 172.21.15.72:0/785158136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077000ac40 con 0x7f0778159360 2023-12-12T09:54:23.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f0776ffd640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0778159360 0x7f077809be90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f076800b420 tx=0x7f076800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:23.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.559+0000 7f075f7fe640 1 -- 172.21.15.72:0/785158136 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768011830 con 0x7f0778159360 2023-12-12T09:54:23.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.560+0000 7f075f7fe640 1 -- 172.21.15.72:0/785158136 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768011990 con 0x7f0778159360 2023-12-12T09:54:23.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.560+0000 7f075f7fe640 1 -- 172.21.15.72:0/785158136 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768011d40 con 0x7f0778159360 2023-12-12T09:54:23.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.560+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0778156ef0 con 0x7f0778159360 2023-12-12T09:54:23.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.560+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07781574b0 con 0x7f0778159360 2023-12-12T09:54:23.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.561+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07780c9470 con 0x7f0778159360 2023-12-12T09:54:23.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.562+0000 7f075f7fe640 1 -- 172.21.15.72:0/785158136 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f076801b460 con 0x7f0778159360 2023-12-12T09:54:23.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.562+0000 7f075f7fe640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f076406ae40 0x7f076406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:23.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.563+0000 7f075f7fe640 1 -- 172.21.15.72:0/785158136 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0768088ff0 con 0x7f0778159360 2023-12-12T09:54:23.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.563+0000 7f07767fc640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f076406ae40 0x7f076406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:23.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.563+0000 7f07767fc640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f076406ae40 0x7f076406d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f077806dbc0 tx=0x7f076c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:23.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.567+0000 7f075f7fe640 1 -- 172.21.15.72:0/785158136 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0768054250 con 0x7f0778159360 2023-12-12T09:54:23.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.758+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0778074450 con 0x7f0778159360 2023-12-12T09:54:23.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.765+0000 7f075f7fe640 1 -- 172.21.15.72:0/785158136 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0768057870 con 0x7f0778159360 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f076406ae40 msgr2=0x7f076406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f076406ae40 0x7f076406d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f077806dbc0 tx=0x7f076c008040 comp rx=0 tx=0).stop 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0778159360 msgr2=0x7f077809be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0778159360 0x7f077809be90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f076800b420 tx=0x7f076800b8e0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 shutdown_connections 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f076406ae40 0x7f076406d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077815d1f0 0x7f07780cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077815cd90 0x7f07780c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 --2- 172.21.15.72:0/785158136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0778159360 0x7f077809be90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 >> 172.21.15.72:0/785158136 conn(0x7f07780cee70 msgr2=0x7f07780cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:23.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 shutdown_connections 2023-12-12T09:54:23.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:23.768+0000 7f077d275640 1 -- 172.21.15.72:0/785158136 wait complete. 2023-12-12T09:54:23.789 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_strays.TestStrays.test_dir_deletion' 2023-12-12T09:54:24.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.010+0000 7f54a639e640 1 Processor -- start 2023-12-12T09:54:24.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.010+0000 7f54a639e640 1 -- start start 2023-12-12T09:54:24.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f54a639e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f54a0143e00 0x7f54a00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f54a639e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54a006c410 0x7f54a015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f54a639e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a00b3bb0 con 0x7f54a0072180 2023-12-12T09:54:24.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f54a639e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a01441e0 con 0x7f54a0143e00 2023-12-12T09:54:24.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f54a639e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a0072680 con 0x7f54a006c410 2023-12-12T09:54:24.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f549ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f54a0143e00 0x7f54a00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f549f7fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f54a0072180 0x7f54a00b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47378/0 (socket says 172.21.15.72:47378) 2023-12-12T09:54:24.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f549f7fe640 1 -- 172.21.15.72:0/7933786 learned_addr learned my addr 172.21.15.72:0/7933786 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:24.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f549ffff640 1 -- 172.21.15.72:0/7933786 >> v2:172.21.15.113:3300/0 conn(0x7f54a006c410 msgr2=0x7f54a015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:24.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.012+0000 7f54a493b640 1 --2- 172.21.15.72:0/7933786 >> v2:172.21.15.113:3300/0 conn(0x7f54a006c410 0x7f54a015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.013+0000 7f549effd640 1 -- 172.21.15.72:0/7933786 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235333265 0 0) 0x7f54a00b3bb0 con 0x7f54a0072180 2023-12-12T09:54:24.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.013+0000 7f549ffff640 1 --2- 172.21.15.72:0/7933786 >> v2:172.21.15.113:3300/0 conn(0x7f54a006c410 0x7f54a015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.013+0000 7f549ffff640 1 -- 172.21.15.72:0/7933786 >> v1:172.21.15.72:6789/0 conn(0x7f54a0072180 legacy=0x7f54a00b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.013+0000 7f549ffff640 1 -- 172.21.15.72:0/7933786 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a00b1e30 con 0x7f54a0143e00 2023-12-12T09:54:24.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.013+0000 7f54a493b640 1 --2- 172.21.15.72:0/7933786 >> v2:172.21.15.113:3300/0 conn(0x7f54a006c410 0x7f54a015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:24.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.014+0000 7f549ffff640 1 --2- 172.21.15.72:0/7933786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0143e00 0x7f54a00b4040 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f54a0072b60 tx=0x7f549002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca146147d7af5696 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.014+0000 7f549effd640 1 -- 172.21.15.72:0/7933786 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490031830 con 0x7f54a0143e00 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.014+0000 7f549effd640 1 -- 172.21.15.72:0/7933786 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5490031990 con 0x7f54a0143e00 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.014+0000 7f549effd640 1 -- 172.21.15.72:0/7933786 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490031d20 con 0x7f54a0143e00 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.015+0000 7f54a639e640 1 -- 172.21.15.72:0/7933786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0143e00 msgr2=0x7f54a00b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.015+0000 7f54a639e640 1 --2- 172.21.15.72:0/7933786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0143e00 0x7f54a00b4040 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f54a0072b60 tx=0x7f549002b8d0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.015+0000 7f54a639e640 1 -- 172.21.15.72:0/7933786 shutdown_connections 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.015+0000 7f54a639e640 1 --2- 172.21.15.72:0/7933786 >> v2:172.21.15.113:3300/0 conn(0x7f54a006c410 0x7f54a015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.015+0000 7f54a639e640 1 --2- 172.21.15.72:0/7933786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0143e00 0x7f54a00b4040 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.015+0000 7f54a639e640 1 -- 172.21.15.72:0/7933786 >> 172.21.15.72:0/7933786 conn(0x7f54a00b0a90 msgr2=0x7f54a00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.015+0000 7f54a639e640 1 -- 172.21.15.72:0/7933786 shutdown_connections 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.015+0000 7f54a639e640 1 -- 172.21.15.72:0/7933786 wait complete. 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.016+0000 7f54a639e640 1 Processor -- start 2023-12-12T09:54:24.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.017+0000 7f54a639e640 1 -- start start 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.017+0000 7f54a639e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54a006c410 0x7f54a0140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.017+0000 7f54a639e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0072180 0x7f54a0141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f54a639e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0143e00 0x7f54a015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f54a639e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a015b020 con 0x7f54a0072180 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f54a639e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a015df50 con 0x7f54a0143e00 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f54a639e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a015bf00 con 0x7f54a006c410 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0072180 0x7f54a0141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0072180 0x7f54a0141200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44188/0 (socket says 172.21.15.72:44188) 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549f7fe640 1 -- 172.21.15.72:0/2370579835 learned_addr learned my addr 172.21.15.72:0/2370579835 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549f7fe640 1 -- 172.21.15.72:0/2370579835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54a006c410 msgr2=0x7f54a0140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549ffff640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54a006c410 0x7f54a0140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549f7fe640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54a006c410 0x7f54a0140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549f7fe640 1 -- 172.21.15.72:0/2370579835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0143e00 msgr2=0x7f54a015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f54a493b640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0143e00 0x7f54a015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549f7fe640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0143e00 0x7f54a015aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.018+0000 7f549f7fe640 1 -- 172.21.15.72:0/2370579835 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490009400 con 0x7f54a0072180 2023-12-12T09:54:24.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.019+0000 7f549f7fe640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0072180 0x7f54a0141200 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f549400c4b0 tx=0x7f549400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:24.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.019+0000 7f549cff9640 1 -- 172.21.15.72:0/2370579835 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54940074e0 con 0x7f54a0072180 2023-12-12T09:54:24.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.019+0000 7f549cff9640 1 -- 172.21.15.72:0/2370579835 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494007660 con 0x7f54a0072180 2023-12-12T09:54:24.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.019+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a015c1e0 con 0x7f54a0072180 2023-12-12T09:54:24.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.019+0000 7f549cff9640 1 -- 172.21.15.72:0/2370579835 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494004a20 con 0x7f54a0072180 2023-12-12T09:54:24.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.019+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a015c7a0 con 0x7f54a0072180 2023-12-12T09:54:24.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.020+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a015e0c0 con 0x7f54a0072180 2023-12-12T09:54:24.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.021+0000 7f549cff9640 1 -- 172.21.15.72:0/2370579835 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5494004b80 con 0x7f54a0072180 2023-12-12T09:54:24.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.022+0000 7f549cff9640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f548c06ae40 0x7f548c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.022+0000 7f549cff9640 1 -- 172.21.15.72:0/2370579835 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5494088590 con 0x7f54a0072180 2023-12-12T09:54:24.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.022+0000 7f549ffff640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f548c06ae40 0x7f548c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.023+0000 7f549ffff640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f548c06ae40 0x7f548c06d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5490036000 tx=0x7f549003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:24.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.026+0000 7f549cff9640 1 -- 172.21.15.72:0/2370579835 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54a015e0c0 con 0x7f54a0072180 2023-12-12T09:54:24.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.288+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_strays.TestStrays.test_dir_deletion"]} v 0) v1 -- 0x7f54a015e0c0 con 0x7f54a0072180 2023-12-12T09:54:24.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.324+0000 7f549cff9640 1 -- 172.21.15.72:0/2370579835 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_strays.TestStrays.test_dir_deletion"]}]=0 v143) v1 ==== 132+0+0 (secure 0 0 0) 0x7f54a015e0c0 con 0x7f54a0072180 2023-12-12T09:54:24.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f548c06ae40 msgr2=0x7f548c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f548c06ae40 0x7f548c06d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5490036000 tx=0x7f549003c040 comp rx=0 tx=0).stop 2023-12-12T09:54:24.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0072180 msgr2=0x7f54a0141200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0072180 0x7f54a0141200 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f549400c4b0 tx=0x7f549400c970 comp rx=0 tx=0).stop 2023-12-12T09:54:24.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 shutdown_connections 2023-12-12T09:54:24.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f548c06ae40 0x7f548c06d300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54a0143e00 0x7f54a015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54a0072180 0x7f54a0141200 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 --2- 172.21.15.72:0/2370579835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54a006c410 0x7f54a0140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 >> 172.21.15.72:0/2370579835 conn(0x7f54a00b0a90 msgr2=0x7f54a006c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:24.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 shutdown_connections 2023-12-12T09:54:24.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.328+0000 7f54a639e640 1 -- 172.21.15.72:0/2370579835 wait complete. 2023-12-12T09:54:24.348 INFO:tasks.cephfs_test_runner:That when deleting a bunch of dentries and the containing ... ok 2023-12-12T09:54:24.348 INFO:tasks.cephfs_test_runner:Starting test: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations). 2023-12-12T09:54:24.348 INFO:tasks.cephfs_test_runner:test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2023-12-12T09:54:24.349 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:24.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.562+0000 7f6e428e1640 1 Processor -- start 2023-12-12T09:54:24.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.562+0000 7f6e428e1640 1 -- start start 2023-12-12T09:54:24.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.564+0000 7f6e428e1640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6e3c0b35c0 0x7f6e3c06a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.564+0000 7f6e428e1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e3c146870 0x7f6e3c06a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.564+0000 7f6e428e1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3c146150 con 0x7f6e3c146320 2023-12-12T09:54:24.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.564+0000 7f6e428e1640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3c0b5060 con 0x7f6e3c0b35c0 2023-12-12T09:54:24.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.564+0000 7f6e428e1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3c0cfa50 con 0x7f6e3c146870 2023-12-12T09:54:24.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.565+0000 7f6e418df640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6e3c0b35c0 0x7f6e3c06a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.565+0000 7f6e418df640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6e3c0b35c0 0x7f6e3c06a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44202/0 (socket says 172.21.15.72:44202) 2023-12-12T09:54:24.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.565+0000 7f6e418df640 1 -- 172.21.15.72:0/2213385534 learned_addr learned my addr 172.21.15.72:0/2213385534 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:24.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.565+0000 7f6e420e0640 1 --2- 172.21.15.72:0/2213385534 >> v2:172.21.15.113:3300/0 conn(0x7f6e3c146870 0x7f6e3c06a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.565+0000 7f6e418df640 1 -- 172.21.15.72:0/2213385534 >> v2:172.21.15.113:3300/0 conn(0x7f6e3c146870 msgr2=0x7f6e3c06a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:24.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.565+0000 7f6e418df640 1 --2- 172.21.15.72:0/2213385534 >> v2:172.21.15.113:3300/0 conn(0x7f6e3c146870 0x7f6e3c06a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.565+0000 7f6e418df640 1 -- 172.21.15.72:0/2213385534 >> v1:172.21.15.113:6789/0 conn(0x7f6e3c146320 legacy=0x7f6e3c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:24.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.565+0000 7f6e418df640 1 -- 172.21.15.72:0/2213385534 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c0cfd30 con 0x7f6e3c0b35c0 2023-12-12T09:54:24.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.566+0000 7f6e418df640 1 --2- 172.21.15.72:0/2213385534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c0b35c0 0x7f6e3c06a460 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c161d10 tx=0x7f6e3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fc0da723da2369 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:24.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e408dd640 1 -- 172.21.15.72:0/2213385534 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38031870 con 0x7f6e3c0b35c0 2023-12-12T09:54:24.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e408dd640 1 -- 172.21.15.72:0/2213385534 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e380319d0 con 0x7f6e3c0b35c0 2023-12-12T09:54:24.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e408dd640 1 -- 172.21.15.72:0/2213385534 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38031d60 con 0x7f6e3c0b35c0 2023-12-12T09:54:24.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e428e1640 1 -- 172.21.15.72:0/2213385534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c0b35c0 msgr2=0x7f6e3c06a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2213385534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c0b35c0 0x7f6e3c06a460 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c161d10 tx=0x7f6e3802b910 comp rx=0 tx=0).stop 2023-12-12T09:54:24.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e428e1640 1 -- 172.21.15.72:0/2213385534 shutdown_connections 2023-12-12T09:54:24.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2213385534 >> v2:172.21.15.113:3300/0 conn(0x7f6e3c146870 0x7f6e3c06a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2213385534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c0b35c0 0x7f6e3c06a460 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e428e1640 1 -- 172.21.15.72:0/2213385534 >> 172.21.15.72:0/2213385534 conn(0x7f6e3c0b1840 msgr2=0x7f6e3c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:24.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e428e1640 1 -- 172.21.15.72:0/2213385534 shutdown_connections 2023-12-12T09:54:24.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.567+0000 7f6e428e1640 1 -- 172.21.15.72:0/2213385534 wait complete. 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.569+0000 7f6e428e1640 1 Processor -- start 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.569+0000 7f6e428e1640 1 -- start start 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.570+0000 7f6e428e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e3c146320 0x7f6e3c144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.570+0000 7f6e428e1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c146870 0x7f6e3c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.570+0000 7f6e428e1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3c15c670 0x7f6e3c152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.570+0000 7f6e428e1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c13dfc0 con 0x7f6e3c146870 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.570+0000 7f6e428e1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c1f6b50 con 0x7f6e3c146320 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.570+0000 7f6e428e1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c1f6e30 con 0x7f6e3c15c670 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c146870 0x7f6e3c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c146870 0x7f6e3c1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44214/0 (socket says 172.21.15.72:44214) 2023-12-12T09:54:24.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 -- 172.21.15.72:0/2680568826 learned_addr learned my addr 172.21.15.72:0/2680568826 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:24.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e420e0640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3c15c670 0x7f6e3c152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 -- 172.21.15.72:0/2680568826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3c15c670 msgr2=0x7f6e3c152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3c15c670 0x7f6e3c152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e418df640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e3c146320 0x7f6e3c144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 -- 172.21.15.72:0/2680568826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e3c146320 msgr2=0x7f6e3c144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e3c146320 0x7f6e3c144310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 -- 172.21.15.72:0/2680568826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380094d0 con 0x7f6e3c146870 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.571+0000 7f6e410de640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c146870 0x7f6e3c1522c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00b3c0 tx=0x7f6e2c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.572+0000 7f6e327fc640 1 -- 172.21.15.72:0/2680568826 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c011830 con 0x7f6e3c146870 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.572+0000 7f6e327fc640 1 -- 172.21.15.72:0/2680568826 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c011990 con 0x7f6e3c146870 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.572+0000 7f6e327fc640 1 -- 172.21.15.72:0/2680568826 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c011d80 con 0x7f6e3c146870 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.572+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c1f7110 con 0x7f6e3c146870 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.572+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3c15cdf0 con 0x7f6e3c146870 2023-12-12T09:54:24.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.573+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3c13e130 con 0x7f6e3c146870 2023-12-12T09:54:24.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.574+0000 7f6e327fc640 1 -- 172.21.15.72:0/2680568826 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e2c01b460 con 0x7f6e3c146870 2023-12-12T09:54:24.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.575+0000 7f6e327fc640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e2006ae40 0x7f6e2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:24.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.575+0000 7f6e327fc640 1 -- 172.21.15.72:0/2680568826 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f6e2c088fa0 con 0x7f6e3c146870 2023-12-12T09:54:24.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.576+0000 7f6e418df640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e2006ae40 0x7f6e2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:24.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.576+0000 7f6e418df640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e2006ae40 0x7f6e2006d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e38036000 tx=0x7f6e3803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:24.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.580+0000 7f6e327fc640 1 -- 172.21.15.72:0/2680568826 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e2c0bbe90 con 0x7f6e3c146870 2023-12-12T09:54:24.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.772+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e3c0673f0 con 0x7f6e3c146870 2023-12-12T09:54:24.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.773+0000 7f6e327fc640 1 -- 172.21.15.72:0/2680568826 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f6e2c054210 con 0x7f6e3c146870 2023-12-12T09:54:24.776 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:24.777 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":43,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:23.311672+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6835/211839795":"2023-12-13T09:54:20.294481+0000","172.21.15.113:6833/211839795":"2023-12-13T09:54:20.294481+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:24.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e2006ae40 msgr2=0x7f6e2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e2006ae40 0x7f6e2006d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e38036000 tx=0x7f6e3803c040 comp rx=0 tx=0).stop 2023-12-12T09:54:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c146870 msgr2=0x7f6e3c1522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c146870 0x7f6e3c1522c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00b3c0 tx=0x7f6e2c00b880 comp rx=0 tx=0).stop 2023-12-12T09:54:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 shutdown_connections 2023-12-12T09:54:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6e2006ae40 0x7f6e2006d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e3c15c670 0x7f6e3c152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e3c146870 0x7f6e3c1522c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 --2- 172.21.15.72:0/2680568826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e3c146320 0x7f6e3c144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:24.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 >> 172.21.15.72:0/2680568826 conn(0x7f6e3c0b1840 msgr2=0x7f6e3c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:24.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 shutdown_connections 2023-12-12T09:54:24.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:24.776+0000 7f6e428e1640 1 -- 172.21.15.72:0/2680568826 wait complete. 2023-12-12T09:54:24.799 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:54:24.799 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:54:25.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.017+0000 7fcfe7d1d640 1 Processor -- start 2023-12-12T09:54:25.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.017+0000 7fcfe7d1d640 1 -- start start 2023-12-12T09:54:25.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.019+0000 7fcfe7d1d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfe00ce7e0 0x7fcfe015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.019+0000 7fcfe7d1d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe0071cd0 con 0x7fcfe015af50 2023-12-12T09:54:25.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.019+0000 7fcfe7d1d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe00719f0 con 0x7fcfe006a460 2023-12-12T09:54:25.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.019+0000 7fcfe7d1d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe00717e0 con 0x7fcfe00ce7e0 2023-12-12T09:54:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.019+0000 7fcfe5ab9640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fcfe015af50 0x7fcfe006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47388/0 (socket says 172.21.15.72:47388) 2023-12-12T09:54:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.019+0000 7fcfe5ab9640 1 -- 172.21.15.72:0/3053234210 learned_addr learned my addr 172.21.15.72:0/3053234210 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:25.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.019+0000 7fcfe52b8640 1 --2- 172.21.15.72:0/3053234210 >> v2:172.21.15.113:3300/0 conn(0x7fcfe00ce7e0 0x7fcfe015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891984955 0 0) 0x7fcfe0071cd0 con 0x7fcfe015af50 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfcc003520 con 0x7fcfe015af50 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3884880605 0 0) 0x7fcfcc003520 con 0x7fcfe015af50 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 >> v2:172.21.15.113:3300/0 conn(0x7fcfe00ce7e0 msgr2=0x7fcfe015ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 --2- 172.21.15.72:0/3053234210 >> v2:172.21.15.113:3300/0 conn(0x7fcfe00ce7e0 0x7fcfe015ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 >> v1:172.21.15.113:6789/0 conn(0x7fcfe006a460 legacy=0x7fcfe00ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe0071ff0 con 0x7fcfe015af50 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fcfdc001670 con 0x7fcfe015af50 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 >> v1:172.21.15.72:6789/0 conn(0x7fcfe015af50 legacy=0x7fcfe006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfcc0039c0 0x7fcfcc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0047c0 0x7fcfcc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc0052a0 0x7fcfcc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00717e0 con 0x7fcfcc0039c0 2023-12-12T09:54:25.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc00c1a0 con 0x7fcfcc0052a0 2023-12-12T09:54:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc00c360 con 0x7fcfcc0047c0 2023-12-12T09:54:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.020+0000 7fcfe5ab9640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfcc0039c0 0x7fcfcc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.021+0000 7fcfe52b8640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0047c0 0x7fcfcc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.021+0000 7fcfe62ba640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc0052a0 0x7fcfcc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.021+0000 7fcfe5ab9640 1 -- 172.21.15.72:0/3053234210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0047c0 msgr2=0x7fcfcc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.021+0000 7fcfe5ab9640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0047c0 0x7fcfcc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.021+0000 7fcfe5ab9640 1 -- 172.21.15.72:0/3053234210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc0052a0 msgr2=0x7fcfcc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.021+0000 7fcfe5ab9640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfcc0052a0 0x7fcfcc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.021+0000 7fcfe5ab9640 1 -- 172.21.15.72:0/3053234210 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc003520 con 0x7fcfcc0039c0 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.021+0000 7fcfe62ba640 1 -- 172.21.15.72:0/3053234210 reap_dead start 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.023+0000 7fcfe5ab9640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfcc0039c0 0x7fcfcc003de0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00c0e0 tx=0x7fcfcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcade5fd06f7c0dc server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.023+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc005ad0 con 0x7fcfcc0039c0 2023-12-12T09:54:25.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.023+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc005c30 con 0x7fcfcc0039c0 2023-12-12T09:54:25.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.023+0000 7fcfe4ab7640 1 -- 172.21.15.72:0/3053234210 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc02ca50 con 0x7fcfcc0039c0 2023-12-12T09:54:25.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.023+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/3053234210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfcc0039c0 msgr2=0x7fcfcc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.023+0000 7fcfe7d1d640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfcc0039c0 0x7fcfcc003de0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00c0e0 tx=0x7fcfcc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.024+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/3053234210 shutdown_connections 2023-12-12T09:54:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.024+0000 7fcfe7d1d640 1 --2- 172.21.15.72:0/3053234210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfcc0039c0 0x7fcfcc003de0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.024+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/3053234210 >> 172.21.15.72:0/3053234210 conn(0x7fcfe00cee70 msgr2=0x7fcfe00cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.024+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/3053234210 shutdown_connections 2023-12-12T09:54:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.024+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/3053234210 wait complete. 2023-12-12T09:54:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.025+0000 7fcfe7d1d640 1 Processor -- start 2023-12-12T09:54:25.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.026+0000 7fcfe7d1d640 1 -- start start 2023-12-12T09:54:25.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.026+0000 7fcfe7d1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0039c0 0x7fcfe009a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.026+0000 7fcfe7d1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfe01543a0 0x7fcfe00c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.026+0000 7fcfe7d1d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe00c3f50 0x7fcfe01f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.026+0000 7fcfe7d1d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe009a370 con 0x7fcfe00c3f50 2023-12-12T09:54:25.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.026+0000 7fcfe7d1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe009b940 con 0x7fcfe01543a0 2023-12-12T09:54:25.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.026+0000 7fcfe7d1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00c43d0 con 0x7fcfcc0039c0 2023-12-12T09:54:25.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe00c3f50 0x7fcfe01f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe00c3f50 0x7fcfe01f0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54106/0 (socket says 172.21.15.72:54106) 2023-12-12T09:54:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 -- 172.21.15.72:0/2668287903 learned_addr learned my addr 172.21.15.72:0/2668287903 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 -- 172.21.15.72:0/2668287903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0039c0 msgr2=0x7fcfe009a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0039c0 0x7fcfe009a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 -- 172.21.15.72:0/2668287903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfe01543a0 msgr2=0x7fcfe00c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfe01543a0 0x7fcfe00c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 -- 172.21.15.72:0/2668287903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe0154800 con 0x7fcfe00c3f50 2023-12-12T09:54:25.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.027+0000 7fcfe62ba640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe00c3f50 0x7fcfe01f0c30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00ce170 tx=0x7fcfd800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.028+0000 7fcfd67fc640 1 -- 172.21.15.72:0/2668287903 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800feb0 con 0x7fcfe00c3f50 2023-12-12T09:54:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.028+0000 7fcfd67fc640 1 -- 172.21.15.72:0/2668287903 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8006d60 con 0x7fcfe00c3f50 2023-12-12T09:54:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.028+0000 7fcfd67fc640 1 -- 172.21.15.72:0/2668287903 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8003bb0 con 0x7fcfe00c3f50 2023-12-12T09:54:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.028+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe00999f0 con 0x7fcfe00c3f50 2023-12-12T09:54:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.028+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe0099cb0 con 0x7fcfe00c3f50 2023-12-12T09:54:25.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.029+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe00cf800 con 0x7fcfe00c3f50 2023-12-12T09:54:25.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.030+0000 7fcfd67fc640 1 -- 172.21.15.72:0/2668287903 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcfd8003d10 con 0x7fcfe00c3f50 2023-12-12T09:54:25.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.031+0000 7fcfd67fc640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfc406ae40 0x7fcfc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.031+0000 7fcfd67fc640 1 -- 172.21.15.72:0/2668287903 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fcfd8088880 con 0x7fcfe00c3f50 2023-12-12T09:54:25.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.031+0000 7fcfe5ab9640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfc406ae40 0x7fcfc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.032+0000 7fcfe5ab9640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfc406ae40 0x7fcfc406d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc034000 tx=0x7fcfdc039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:25.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.036+0000 7fcfd67fc640 1 -- 172.21.15.72:0/2668287903 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfd8053af0 con 0x7fcfe00c3f50 2023-12-12T09:54:25.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.228+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfe009a4b0 con 0x7fcfe00c3f50 2023-12-12T09:54:25.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.230+0000 7fcfd67fc640 1 -- 172.21.15.72:0/2668287903 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfe009a4b0 con 0x7fcfe00c3f50 2023-12-12T09:54:25.232 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:54:25.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfc406ae40 msgr2=0x7fcfc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfc406ae40 0x7fcfc406d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc034000 tx=0x7fcfdc039040 comp rx=0 tx=0).stop 2023-12-12T09:54:25.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe00c3f50 msgr2=0x7fcfe01f0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe00c3f50 0x7fcfe01f0c30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00ce170 tx=0x7fcfd800fc00 comp rx=0 tx=0).stop 2023-12-12T09:54:25.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 shutdown_connections 2023-12-12T09:54:25.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcfc406ae40 0x7fcfc406d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe00c3f50 0x7fcfe01f0c30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfe01543a0 0x7fcfe00c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 --2- 172.21.15.72:0/2668287903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcfcc0039c0 0x7fcfe009a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 >> 172.21.15.72:0/2668287903 conn(0x7fcfe00cee70 msgr2=0x7fcfe0070850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:25.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 shutdown_connections 2023-12-12T09:54:25.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.232+0000 7fcfe7d1d640 1 -- 172.21.15.72:0/2668287903 wait complete. 2023-12-12T09:54:25.250 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit' 2023-12-12T09:54:25.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.452+0000 7f80bdb61640 1 Processor -- start 2023-12-12T09:54:25.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.453+0000 7f80bdb61640 1 -- start start 2023-12-12T09:54:25.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.454+0000 7f80bdb61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80b806cb00 0x7f80b80cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.454+0000 7f80bdb61640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b806a5e0 con 0x7f80b80ac860 2023-12-12T09:54:25.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.454+0000 7f80bdb61640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b80ce1e0 con 0x7f80b80cd980 2023-12-12T09:54:25.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.454+0000 7f80bdb61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b806a8a0 con 0x7f80b806cb00 2023-12-12T09:54:25.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.454+0000 7f80b77fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f80b80ac860 0x7f80b80acc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35684/0 (socket says 172.21.15.72:35684) 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.454+0000 7f80b77fe640 1 -- 172.21.15.72:0/1782991621 learned_addr learned my addr 172.21.15.72:0/1782991621 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876824666 0 0) 0x7f80b806a5e0 con 0x7f80b80ac860 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a4003520 con 0x7f80b80ac860 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b6ffd640 1 --2- 172.21.15.72:0/1782991621 >> v2:172.21.15.113:3300/0 conn(0x7f80b806cb00 0x7f80b80cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224031640 0 0) 0x7f80a4003520 con 0x7f80b80ac860 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 >> v2:172.21.15.113:3300/0 conn(0x7f80b806cb00 msgr2=0x7f80b80cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 --2- 172.21.15.72:0/1782991621 >> v2:172.21.15.113:3300/0 conn(0x7f80b806cb00 0x7f80b80cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 >> v1:172.21.15.113:6789/0 conn(0x7f80b80cd980 legacy=0x7f80b806b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b806ab80 con 0x7f80b80ac860 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f80b00019f0 con 0x7f80b80ac860 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 >> v1:172.21.15.72:6789/0 conn(0x7f80b80ac860 legacy=0x7f80b80acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80a4003970 0x7f80a4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a40047c0 0x7f80a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a4005180 0x7f80a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b806a8a0 con 0x7f80a4003970 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a400c0b0 con 0x7f80a4005180 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.455+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a400c270 con 0x7f80a40047c0 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.456+0000 7f80b77fe640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80a4003970 0x7f80a4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.456+0000 7f80b77fe640 1 -- 172.21.15.72:0/1782991621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a40047c0 msgr2=0x7f80a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.456+0000 7f80b77fe640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a40047c0 0x7f80a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.456+0000 7f80b77fe640 1 -- 172.21.15.72:0/1782991621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a4005180 msgr2=0x7f80a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.456+0000 7f80b77fe640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a4005180 0x7f80a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.456+0000 7f80b77fe640 1 -- 172.21.15.72:0/1782991621 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4003520 con 0x7f80a4003970 2023-12-12T09:54:25.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.456+0000 7f80b7fff640 1 -- 172.21.15.72:0/1782991621 reap_dead start 2023-12-12T09:54:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.457+0000 7f80b77fe640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80a4003970 0x7f80a4003d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f80b0002230 tx=0x7f80a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1772b6237a5d2a77 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.457+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0034630 con 0x7f80a4003970 2023-12-12T09:54:25.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.457+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b002f070 con 0x7f80a4003970 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.457+0000 7f80b67fc640 1 -- 172.21.15.72:0/1782991621 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0034a30 con 0x7f80a4003970 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.457+0000 7f80bdb61640 1 -- 172.21.15.72:0/1782991621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80a4003970 msgr2=0x7f80a4003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.457+0000 7f80bdb61640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80a4003970 0x7f80a4003d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f80b0002230 tx=0x7f80a40057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.458+0000 7f80bdb61640 1 -- 172.21.15.72:0/1782991621 shutdown_connections 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.458+0000 7f80bdb61640 1 --2- 172.21.15.72:0/1782991621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80a4003970 0x7f80a4003d70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.458+0000 7f80bdb61640 1 -- 172.21.15.72:0/1782991621 >> 172.21.15.72:0/1782991621 conn(0x7f80b80cf540 msgr2=0x7f80b80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.458+0000 7f80bdb61640 1 -- 172.21.15.72:0/1782991621 shutdown_connections 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.458+0000 7f80bdb61640 1 -- 172.21.15.72:0/1782991621 wait complete. 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.459+0000 7f80bdb61640 1 Processor -- start 2023-12-12T09:54:25.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.460+0000 7f80bdb61640 1 -- start start 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.460+0000 7f80bdb61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a4003970 0x7f80b81f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80bdb61640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8156860 0x7f80b8148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80bdb61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b806afa0 0x7f80b806b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80bdb61640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b806b960 con 0x7f80b8156860 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80bdb61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b815fd90 con 0x7f80a4003970 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80bdb61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8160070 con 0x7f80b806afa0 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8156860 0x7f80b8148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8156860 0x7f80b8148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54120/0 (socket says 172.21.15.72:54120) 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 -- 172.21.15.72:0/2077716888 learned_addr learned my addr 172.21.15.72:0/2077716888 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:25.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 -- 172.21.15.72:0/2077716888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b806afa0 msgr2=0x7f80b806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b806afa0 0x7f80b806b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 -- 172.21.15.72:0/2077716888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a4003970 msgr2=0x7f80b81f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a4003970 0x7f80b81f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 -- 172.21.15.72:0/2077716888 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b81f6100 con 0x7f80b8156860 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.461+0000 7f80b6ffd640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8156860 0x7f80b8148eb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f80a000d530 tx=0x7f80a000d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.462+0000 7f80affff640 1 -- 172.21.15.72:0/2077716888 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a00125f0 con 0x7f80b8156860 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.462+0000 7f80affff640 1 -- 172.21.15.72:0/2077716888 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a0012750 con 0x7f80b8156860 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.462+0000 7f80affff640 1 -- 172.21.15.72:0/2077716888 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0012b00 con 0x7f80b8156860 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.462+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b81f63e0 con 0x7f80b8156860 2023-12-12T09:54:25.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.462+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b8156ce0 con 0x7f80b8156860 2023-12-12T09:54:25.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.463+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b80745d0 con 0x7f80b8156860 2023-12-12T09:54:25.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.464+0000 7f80affff640 1 -- 172.21.15.72:0/2077716888 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80a0012c60 con 0x7f80b8156860 2023-12-12T09:54:25.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.465+0000 7f80affff640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f809806ae40 0x7f809806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:25.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.465+0000 7f80affff640 1 -- 172.21.15.72:0/2077716888 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f80a0089780 con 0x7f80b8156860 2023-12-12T09:54:25.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.465+0000 7f80b77fe640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f809806ae40 0x7f809806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:25.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.466+0000 7f80b77fe640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f809806ae40 0x7f809806d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f80b0034000 tx=0x7f80b0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:25.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.469+0000 7f80affff640 1 -- 172.21.15.72:0/2077716888 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80a0054a80 con 0x7f80b8156860 2023-12-12T09:54:25.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:25.728+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit"]} v 0) v1 -- 0x7f80b80aed30 con 0x7f80b8156860 2023-12-12T09:54:26.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.337+0000 7f80affff640 1 -- 172.21.15.72:0/2077716888 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit"]}]=0 v145) v1 ==== 136+0+0 (secure 0 0 0) 0x7f80a00580a0 con 0x7f80b8156860 2023-12-12T09:54:26.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f809806ae40 msgr2=0x7f809806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f809806ae40 0x7f809806d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f80b0034000 tx=0x7f80b0009030 comp rx=0 tx=0).stop 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8156860 msgr2=0x7f80b8148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8156860 0x7f80b8148eb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f80a000d530 tx=0x7f80a000d9f0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 shutdown_connections 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f809806ae40 0x7f809806d300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b806afa0 0x7f80b806b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8156860 0x7f80b8148eb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 --2- 172.21.15.72:0/2077716888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a4003970 0x7f80b81f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.340+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 >> 172.21.15.72:0/2077716888 conn(0x7f80b80cf540 msgr2=0x7f80b806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.341+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 shutdown_connections 2023-12-12T09:54:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.341+0000 7f80bdb61640 1 -- 172.21.15.72:0/2077716888 wait complete. 2023-12-12T09:54:26.363 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-12T09:54:26.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.579+0000 7f0da2fc9640 1 Processor -- start 2023-12-12T09:54:26.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.579+0000 7f0da2fc9640 1 -- start start 2023-12-12T09:54:26.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.580+0000 7f0da2fc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c0b4550 0x7f0d9c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:26.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.580+0000 7f0da2fc9640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0d9c0d0140 0x7f0d9c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:26.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.580+0000 7f0da2fc9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9c0ce6e0 con 0x7f0d9c15e720 2023-12-12T09:54:26.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.580+0000 7f0da2fc9640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9c0d21f0 con 0x7f0d9c0d0140 2023-12-12T09:54:26.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.580+0000 7f0da2fc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c0ce9c0 con 0x7f0d9c0b4550 2023-12-12T09:54:26.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da1566640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0d9c0d0140 0x7f0d9c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:26.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da1566640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0d9c0d0140 0x7f0d9c15cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54128/0 (socket says 172.21.15.72:54128) 2023-12-12T09:54:26.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da1566640 1 -- 172.21.15.72:0/1432687886 learned_addr learned my addr 172.21.15.72:0/1432687886 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:26.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da0d65640 1 --2- 172.21.15.72:0/1432687886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c0b4550 0x7f0d9c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:26.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da1566640 1 -- 172.21.15.72:0/1432687886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c0b4550 msgr2=0x7f0d9c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:26.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da1566640 1 --2- 172.21.15.72:0/1432687886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c0b4550 0x7f0d9c0cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da1566640 1 -- 172.21.15.72:0/1432687886 >> v1:172.21.15.113:6789/0 conn(0x7f0d9c15e720 legacy=0x7f0d9c0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:26.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da1566640 1 -- 172.21.15.72:0/1432687886 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c0ceca0 con 0x7f0d9c0d0140 2023-12-12T09:54:26.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.581+0000 7f0da0d65640 1 --2- 172.21.15.72:0/1432687886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c0b4550 0x7f0d9c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:26.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.582+0000 7f0da1566640 1 --2- 172.21.15.72:0/1432687886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0d0140 0x7f0d9c15cd40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009870 tx=0x7f0d9402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be97a2b3b019f4b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:26.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0d9b7fe640 1 -- 172.21.15.72:0/1432687886 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94031640 con 0x7f0d9c0d0140 2023-12-12T09:54:26.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0d9b7fe640 1 -- 172.21.15.72:0/1432687886 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d940317a0 con 0x7f0d9c0d0140 2023-12-12T09:54:26.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0d9b7fe640 1 -- 172.21.15.72:0/1432687886 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94031b30 con 0x7f0d9c0d0140 2023-12-12T09:54:26.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0da2fc9640 1 -- 172.21.15.72:0/1432687886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0d0140 msgr2=0x7f0d9c15cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/1432687886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0d0140 0x7f0d9c15cd40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009870 tx=0x7f0d9402b8f0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0da2fc9640 1 -- 172.21.15.72:0/1432687886 shutdown_connections 2023-12-12T09:54:26.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/1432687886 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0d0140 0x7f0d9c15cd40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/1432687886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c0b4550 0x7f0d9c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0da2fc9640 1 -- 172.21.15.72:0/1432687886 >> 172.21.15.72:0/1432687886 conn(0x7f0d9c09c430 msgr2=0x7f0d9c0d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:26.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.583+0000 7f0da2fc9640 1 -- 172.21.15.72:0/1432687886 shutdown_connections 2023-12-12T09:54:26.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.584+0000 7f0da2fc9640 1 -- 172.21.15.72:0/1432687886 wait complete. 2023-12-12T09:54:26.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.585+0000 7f0da2fc9640 1 Processor -- start 2023-12-12T09:54:26.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.585+0000 7f0da2fc9640 1 -- start start 2023-12-12T09:54:26.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.586+0000 7f0da2fc9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0b4550 0x7f0d9c14a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:26.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.586+0000 7f0da2fc9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c0d0140 0x7f0d9c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:26.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.586+0000 7f0da2fc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c15e720 0x7f0d9c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.586+0000 7f0da2fc9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c1f4c30 con 0x7f0d9c0b4550 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.586+0000 7f0da2fc9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c0c6f80 con 0x7f0d9c0d0140 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.586+0000 7f0da2fc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c1f3fb0 con 0x7f0d9c15e720 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.586+0000 7f0da0d65640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0b4550 0x7f0d9c14a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0da0d65640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0b4550 0x7f0d9c14a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54140/0 (socket says 172.21.15.72:54140) 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0da0d65640 1 -- 172.21.15.72:0/2217419375 learned_addr learned my addr 172.21.15.72:0/2217419375 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0d9bfff640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c0d0140 0x7f0d9c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0da0d65640 1 -- 172.21.15.72:0/2217419375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c15e720 msgr2=0x7f0d9c0b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0da0d65640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c15e720 0x7f0d9c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0da0d65640 1 -- 172.21.15.72:0/2217419375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c0d0140 msgr2=0x7f0d9c0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0da0d65640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c0d0140 0x7f0d9c0b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0da0d65640 1 -- 172.21.15.72:0/2217419375 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d940094d0 con 0x7f0d9c0b4550 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0da0d65640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0b4550 0x7f0d9c14a320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00cf60 tx=0x7f0d8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.587+0000 7f0d997fa640 1 -- 172.21.15.72:0/2217419375 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00d840 con 0x7f0d9c0b4550 2023-12-12T09:54:26.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.588+0000 7f0d997fa640 1 -- 172.21.15.72:0/2217419375 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c00d9a0 con 0x7f0d9c0b4550 2023-12-12T09:54:26.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.588+0000 7f0d997fa640 1 -- 172.21.15.72:0/2217419375 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00dd30 con 0x7f0d9c0b4550 2023-12-12T09:54:26.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.588+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9c1f4290 con 0x7f0d9c0b4550 2023-12-12T09:54:26.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.588+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c1f4850 con 0x7f0d9c0b4550 2023-12-12T09:54:26.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.589+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9c1f4d70 con 0x7f0d9c0b4550 2023-12-12T09:54:26.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.590+0000 7f0d997fa640 1 -- 172.21.15.72:0/2217419375 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d8c015440 con 0x7f0d9c0b4550 2023-12-12T09:54:26.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.591+0000 7f0d997fa640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0d8806ae40 0x7f0d8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:26.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.591+0000 7f0d997fa640 1 -- 172.21.15.72:0/2217419375 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0d8c089e10 con 0x7f0d9c0b4550 2023-12-12T09:54:26.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.591+0000 7f0d9bfff640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0d8806ae40 0x7f0d8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:26.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.592+0000 7f0d9bfff640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0d8806ae40 0x7f0d8806d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0c9c00 tx=0x7f0d90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:26.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.595+0000 7f0d997fa640 1 -- 172.21.15.72:0/2217419375 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d9c1f4d70 con 0x7f0d9c0b4550 2023-12-12T09:54:26.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.785+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0d9c1f4d70 con 0x7f0d9c0b4550 2023-12-12T09:54:26.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.791+0000 7f0d997fa640 1 -- 172.21.15.72:0/2217419375 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0d9c1f4d70 con 0x7f0d9c0b4550 2023-12-12T09:54:26.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0d8806ae40 msgr2=0x7f0d8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0d8806ae40 0x7f0d8806d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0c9c00 tx=0x7f0d90008040 comp rx=0 tx=0).stop 2023-12-12T09:54:26.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0b4550 msgr2=0x7f0d9c14a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0b4550 0x7f0d9c14a320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00cf60 tx=0x7f0d8c00cf90 comp rx=0 tx=0).stop 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 shutdown_connections 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0d8806ae40 0x7f0d8806d300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d9c15e720 0x7f0d9c0b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9c0d0140 0x7f0d9c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 --2- 172.21.15.72:0/2217419375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d9c0b4550 0x7f0d9c14a320 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 >> 172.21.15.72:0/2217419375 conn(0x7f0d9c09c430 msgr2=0x7f0d9c0d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.794+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 shutdown_connections 2023-12-12T09:54:26.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:26.795+0000 7f0da2fc9640 1 -- 172.21.15.72:0/2217419375 wait complete. 2023-12-12T09:54:26.816 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-12T09:54:26.816 DEBUG:teuthology.orchestra.run.smithi072:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-12T09:54:26.930 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-12T09:54:27.049 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-12T09:54:27.049 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:27.049 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:27.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054258,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-12T09:54:27.109 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:54:27.109 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:27.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054290,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-12T09:54:27.169 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:27.169 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:27.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054258,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-12T09:54:27.229 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:54:27.229 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:27.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054290,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-12T09:54:27.290 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-12T09:54:27.290 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:27.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.510+0000 7f7c01648640 1 Processor -- start 2023-12-12T09:54:27.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.510+0000 7f7c01648640 1 -- start start 2023-12-12T09:54:27.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.512+0000 7f7c01648640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bfc0cdec0 0x7f7bfc1441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:27.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.512+0000 7f7c01648640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc06d520 con 0x7f7bfc143990 2023-12-12T09:54:27.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.512+0000 7f7c01648640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc0ce320 con 0x7f7bfc06a460 2023-12-12T09:54:27.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.512+0000 7f7c01648640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc143d70 con 0x7f7bfc0cdec0 2023-12-12T09:54:27.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.512+0000 7f7bfaffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f7bfc143990 0x7f7bfc071be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35686/0 (socket says 172.21.15.72:35686) 2023-12-12T09:54:27.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.512+0000 7f7bfaffd640 1 -- 172.21.15.72:0/3775362807 learned_addr learned my addr 172.21.15.72:0/3775362807 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:27.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bf3fff640 1 --2- 172.21.15.72:0/3775362807 >> v2:172.21.15.113:3300/0 conn(0x7f7bfc0cdec0 0x7f7bfc1441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802334234 0 0) 0x7f7bfc06d520 con 0x7f7bfc143990 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be8003520 con 0x7f7bfc143990 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2774058329 0 0) 0x7f7be8003520 con 0x7f7bfc143990 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 >> v2:172.21.15.113:3300/0 conn(0x7f7bfc0cdec0 msgr2=0x7f7bfc1441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 --2- 172.21.15.72:0/3775362807 >> v2:172.21.15.113:3300/0 conn(0x7f7bfc0cdec0 0x7f7bfc1441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 >> v1:172.21.15.113:6789/0 conn(0x7f7bfc06a460 legacy=0x7f7bfc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc0723c0 con 0x7f7bfc143990 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f7bec001670 con 0x7f7bfc143990 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 >> v1:172.21.15.72:6789/0 conn(0x7f7bfc143990 legacy=0x7f7bfc071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7be80039c0 0x7f7be8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7be80047c0 0x7f7be8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80051d0 0x7f7be800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc143d70 con 0x7f7be80047c0 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be800c120 con 0x7f7be80039c0 2023-12-12T09:54:27.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.513+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be800c2e0 con 0x7f7be80051d0 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bf3fff640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7be80047c0 0x7f7be8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bfaffd640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7be80039c0 0x7f7be8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bfb7fe640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80051d0 0x7f7be800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bf3fff640 1 -- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80051d0 msgr2=0x7f7be800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bf3fff640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80051d0 0x7f7be800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bf3fff640 1 -- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7be80039c0 msgr2=0x7f7be8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bf3fff640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7be80039c0 0x7f7be8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bf3fff640 1 -- 172.21.15.72:0/3775362807 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be8003520 con 0x7f7be80047c0 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bfb7fe640 1 -- 172.21.15.72:0/3775362807 reap_dead start 2023-12-12T09:54:27.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.514+0000 7f7bfaffd640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7be80039c0 0x7f7be8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:54:27.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.515+0000 7f7bf3fff640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7be80047c0 0x7f7be8004c40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ba30 tx=0x7f7be80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c0f858aae140c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:27.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4032640 con 0x7f7be80047c0 2023-12-12T09:54:27.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be40327a0 con 0x7f7be80047c0 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7bfa7fc640 1 -- 172.21.15.72:0/3775362807 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4032b30 con 0x7f7be80047c0 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7c01648640 1 -- 172.21.15.72:0/3775362807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7be80047c0 msgr2=0x7f7be8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7c01648640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7be80047c0 0x7f7be8004c40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ba30 tx=0x7f7be80057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7c01648640 1 -- 172.21.15.72:0/3775362807 shutdown_connections 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7c01648640 1 --2- 172.21.15.72:0/3775362807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7be80047c0 0x7f7be8004c40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7c01648640 1 -- 172.21.15.72:0/3775362807 >> 172.21.15.72:0/3775362807 conn(0x7f7bfc0cfe30 msgr2=0x7f7bfc0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7c01648640 1 -- 172.21.15.72:0/3775362807 shutdown_connections 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.516+0000 7f7c01648640 1 -- 172.21.15.72:0/3775362807 wait complete. 2023-12-12T09:54:27.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.518+0000 7f7c01648640 1 Processor -- start 2023-12-12T09:54:27.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.518+0000 7f7c01648640 1 -- start start 2023-12-12T09:54:27.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.519+0000 7f7c01648640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80047c0 0x7f7bfc15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:27.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.519+0000 7f7c01648640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bfc0c57e0 0x7f7bfc145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:27.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.519+0000 7f7c01648640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bfc1461e0 0x7f7bfc1f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:27.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.519+0000 7f7c01648640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc146660 con 0x7f7bfc1461e0 2023-12-12T09:54:27.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.519+0000 7f7c01648640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc13c350 con 0x7f7bfc0c57e0 2023-12-12T09:54:27.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.519+0000 7f7c01648640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc15d280 con 0x7f7be80047c0 2023-12-12T09:54:27.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bfc1461e0 0x7f7bfc1f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:27.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bfc1461e0 0x7f7bfc1f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54150/0 (socket says 172.21.15.72:54150) 2023-12-12T09:54:27.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 -- 172.21.15.72:0/3597398898 learned_addr learned my addr 172.21.15.72:0/3597398898 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:27.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfaffd640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80047c0 0x7f7bfc15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:27.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bf3fff640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bfc0c57e0 0x7f7bfc145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:27.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 -- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80047c0 msgr2=0x7f7bfc15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:27.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80047c0 0x7f7bfc15cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 -- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bfc0c57e0 msgr2=0x7f7bfc145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:27.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bfc0c57e0 0x7f7bfc145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 -- 172.21.15.72:0/3597398898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc13ba50 con 0x7f7bfc1461e0 2023-12-12T09:54:27.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bf3fff640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bfc0c57e0 0x7f7bfc145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:27.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.520+0000 7f7bfb7fe640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bfc1461e0 0x7f7bfc1f6210 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc070f40 tx=0x7f7bf400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:27.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.521+0000 7f7bf37fe640 1 -- 172.21.15.72:0/3597398898 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4003b70 con 0x7f7bfc1461e0 2023-12-12T09:54:27.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.521+0000 7f7bf37fe640 1 -- 172.21.15.72:0/3597398898 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf4003cd0 con 0x7f7bfc1461e0 2023-12-12T09:54:27.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.521+0000 7f7bf37fe640 1 -- 172.21.15.72:0/3597398898 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4005290 con 0x7f7bfc1461e0 2023-12-12T09:54:27.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.521+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc13bd30 con 0x7f7bfc1461e0 2023-12-12T09:54:27.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.521+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc13c8d0 con 0x7f7bfc1461e0 2023-12-12T09:54:27.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.522+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc13cac0 con 0x7f7bfc1461e0 2023-12-12T09:54:27.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.523+0000 7f7bf37fe640 1 -- 172.21.15.72:0/3597398898 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bf40053f0 con 0x7f7bfc1461e0 2023-12-12T09:54:27.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.524+0000 7f7bf37fe640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bdc06ae40 0x7f7bdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:27.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.524+0000 7f7bf37fe640 1 -- 172.21.15.72:0/3597398898 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7bf4088da0 con 0x7f7bfc1461e0 2023-12-12T09:54:27.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.524+0000 7f7bfaffd640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bdc06ae40 0x7f7bdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:27.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.525+0000 7f7bfaffd640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bdc06ae40 0x7f7bdc06d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7bec001bb0 tx=0x7f7bec001b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:27.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.528+0000 7f7bf37fe640 1 -- 172.21.15.72:0/3597398898 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bf4053f70 con 0x7f7bfc1461e0 2023-12-12T09:54:27.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.757+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bfc0745d0 con 0x7f7bfc1461e0 2023-12-12T09:54:27.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.758+0000 7f7bf37fe640 1 -- 172.21.15.72:0/3597398898 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f7bf4018070 con 0x7f7bfc1461e0 2023-12-12T09:54:27.760 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:27.760 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2973303779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2973303779},{"type":"v1","addr":"172.21.15.72:6837","nonce":2973303779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/4110188890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4110188890},{"type":"v1","addr":"172.21.15.113:6834","nonce":4110188890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/4188101098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":4188101098},{"type":"v1","addr":"172.21.15.113:6835","nonce":4188101098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5322,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/4099456208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4099456208},{"type":"v1","addr":"172.21.15.72:6836","nonce":4099456208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-12-12T09:54:27.761 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 18 2023-12-12T09:54:27.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bdc06ae40 msgr2=0x7f7bdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:27.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bdc06ae40 0x7f7bdc06d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7bec001bb0 tx=0x7f7bec001b20 comp rx=0 tx=0).stop 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bfc1461e0 msgr2=0x7f7bfc1f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bfc1461e0 0x7f7bfc1f6210 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc070f40 tx=0x7f7bf400dc00 comp rx=0 tx=0).stop 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 shutdown_connections 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7bdc06ae40 0x7f7bdc06d300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bfc1461e0 0x7f7bfc1f6210 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bfc0c57e0 0x7f7bfc145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 --2- 172.21.15.72:0/3597398898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7be80047c0 0x7f7bfc15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 >> 172.21.15.72:0/3597398898 conn(0x7f7bfc0cfe30 msgr2=0x7f7bfc15f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 shutdown_connections 2023-12-12T09:54:27.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:27.763+0000 7f7c01648640 1 -- 172.21.15.72:0/3597398898 wait complete. 2023-12-12T09:54:27.783 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-12T09:54:27.783 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-12T09:54:27.783 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-12T09:54:27.783 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:54:27.783 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-12T09:54:27.783 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-12T09:54:27.784 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-12T09:54:27.784 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:54:27.784 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-12T09:54:27.784 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-12T09:54:27.784 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-12T09:54:27.784 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:54:27.784 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-12T09:54:27.784 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-12T09:54:27.785 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-12T09:54:27.785 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:54:27.801 INFO:tasks.ceph.mds.a:Stopped 2023-12-12T09:54:27.801 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-12T09:54:27.805 INFO:tasks.ceph.mds.c:Stopped 2023-12-12T09:54:27.805 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-12T09:54:27.813 INFO:tasks.ceph.mds.b:Stopped 2023-12-12T09:54:27.813 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-12T09:54:27.814 INFO:tasks.ceph.mds.d:Stopped 2023-12-12T09:54:27.814 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-12T09:54:27.853 INFO:tasks.ceph.mds.a:Started 2023-12-12T09:54:27.854 DEBUG:teuthology.parallel:result is None 2023-12-12T09:54:27.861 INFO:tasks.ceph.mds.c:Started 2023-12-12T09:54:27.861 DEBUG:teuthology.parallel:result is None 2023-12-12T09:54:27.862 INFO:tasks.ceph.mds.b:Started 2023-12-12T09:54:27.862 DEBUG:teuthology.parallel:result is None 2023-12-12T09:54:27.871 INFO:tasks.ceph.mds.d:Started 2023-12-12T09:54:27.871 DEBUG:teuthology.parallel:result is None 2023-12-12T09:54:27.872 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-12T09:54:28.013 INFO:tasks.ceph.mds.a.smithi072.stdout:starting mds.a at 2023-12-12T09:54:28.013 INFO:tasks.ceph.mds.c.smithi072.stdout:starting mds.c at 2023-12-12T09:54:28.026 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-12-12T09:54:28.027 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-12-12T09:54:28.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.059+0000 7ff014a4a640 1 Processor -- start 2023-12-12T09:54:28.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.059+0000 7ff014a4a640 1 -- start start 2023-12-12T09:54:28.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff014a4a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff01015e5d0 0x7ff01006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff014a4a640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff010071260 0x7ff0100cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff014a4a640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff010071f90 con 0x7ff0100ae200 2023-12-12T09:54:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff014a4a640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff010072270 con 0x7ff010071260 2023-12-12T09:54:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff014a4a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff010070070 con 0x7ff01015e5d0 2023-12-12T09:54:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff00e59c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff0100ae200 0x7ff010070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35714/0 (socket says 172.21.15.72:35714) 2023-12-12T09:54:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff00e59c640 1 -- 172.21.15.72:0/3857882032 learned_addr learned my addr 172.21.15.72:0/3857882032 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff00f59e640 1 --2- 172.21.15.72:0/3857882032 >> v2:172.21.15.72:3300/0 conn(0x7ff010071260 0x7ff0100cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff00ed9d640 1 --2- 172.21.15.72:0/3857882032 >> v2:172.21.15.113:3300/0 conn(0x7ff01015e5d0 0x7ff01006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff00f59e640 1 -- 172.21.15.72:0/3857882032 >> v2:172.21.15.113:3300/0 conn(0x7ff01015e5d0 msgr2=0x7ff01006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:28.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff00f59e640 1 --2- 172.21.15.72:0/3857882032 >> v2:172.21.15.113:3300/0 conn(0x7ff01015e5d0 0x7ff01006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff00f59e640 1 -- 172.21.15.72:0/3857882032 >> v1:172.21.15.72:6789/0 conn(0x7ff0100ae200 legacy=0x7ff010070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.061+0000 7ff00f59e640 1 -- 172.21.15.72:0/3857882032 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff010070350 con 0x7ff010071260 2023-12-12T09:54:28.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.062+0000 7ff00f59e640 1 --2- 172.21.15.72:0/3857882032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff010071260 0x7ff0100cded0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff01006bb50 tx=0x7ff00802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc7040a7e9a4155 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:28.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff00dd9b640 1 -- 172.21.15.72:0/3857882032 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00802cbe0 con 0x7ff010071260 2023-12-12T09:54:28.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff00dd9b640 1 -- 172.21.15.72:0/3857882032 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00802cd40 con 0x7ff010071260 2023-12-12T09:54:28.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff00dd9b640 1 -- 172.21.15.72:0/3857882032 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008034660 con 0x7ff010071260 2023-12-12T09:54:28.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff014a4a640 1 -- 172.21.15.72:0/3857882032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff010071260 msgr2=0x7ff0100cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff014a4a640 1 --2- 172.21.15.72:0/3857882032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff010071260 0x7ff0100cded0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff01006bb50 tx=0x7ff00802be30 comp rx=0 tx=0).stop 2023-12-12T09:54:28.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff014a4a640 1 -- 172.21.15.72:0/3857882032 shutdown_connections 2023-12-12T09:54:28.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff014a4a640 1 --2- 172.21.15.72:0/3857882032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff010071260 0x7ff0100cded0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff014a4a640 1 --2- 172.21.15.72:0/3857882032 >> v2:172.21.15.113:3300/0 conn(0x7ff01015e5d0 0x7ff01006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.063+0000 7ff014a4a640 1 -- 172.21.15.72:0/3857882032 >> 172.21.15.72:0/3857882032 conn(0x7ff0100b0860 msgr2=0x7ff0100b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:28.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.064+0000 7ff014a4a640 1 -- 172.21.15.72:0/3857882032 shutdown_connections 2023-12-12T09:54:28.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.064+0000 7ff014a4a640 1 -- 172.21.15.72:0/3857882032 wait complete. 2023-12-12T09:54:28.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.065+0000 7ff014a4a640 1 Processor -- start 2023-12-12T09:54:28.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.065+0000 7ff014a4a640 1 -- start start 2023-12-12T09:54:28.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.066+0000 7ff014a4a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff010071260 0x7ff01015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.066+0000 7ff014a4a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100ae200 0x7ff01015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.066+0000 7ff014a4a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff01015e5d0 0x7ff010157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.066+0000 7ff014a4a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0101f2e20 con 0x7ff01015e5d0 2023-12-12T09:54:28.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.066+0000 7ff014a4a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01015c100 con 0x7ff0100ae200 2023-12-12T09:54:28.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.066+0000 7ff014a4a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01015c380 con 0x7ff010071260 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00f59e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff01015e5d0 0x7ff010157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00f59e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff01015e5d0 0x7ff010157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54196/0 (socket says 172.21.15.72:54196) 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00f59e640 1 -- 172.21.15.72:0/1530137124 learned_addr learned my addr 172.21.15.72:0/1530137124 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00e59c640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100ae200 0x7ff01015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00ed9d640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff010071260 0x7ff01015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00f59e640 1 -- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff010071260 msgr2=0x7ff01015d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00f59e640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff010071260 0x7ff01015d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00f59e640 1 -- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100ae200 msgr2=0x7ff01015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00f59e640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100ae200 0x7ff01015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00f59e640 1 -- 172.21.15.72:0/1530137124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008009a70 con 0x7ff01015e5d0 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.067+0000 7ff00e59c640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100ae200 0x7ff01015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:54:28.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.068+0000 7ff00f59e640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff01015e5d0 0x7ff010157210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff008009e80 tx=0x7ff008004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:28.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.068+0000 7feff77fe640 1 -- 172.21.15.72:0/1530137124 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008040650 con 0x7ff01015e5d0 2023-12-12T09:54:28.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.068+0000 7feff77fe640 1 -- 172.21.15.72:0/1530137124 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0080407d0 con 0x7ff01015e5d0 2023-12-12T09:54:28.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.068+0000 7feff77fe640 1 -- 172.21.15.72:0/1530137124 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008040b80 con 0x7ff01015e5d0 2023-12-12T09:54:28.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.068+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01015c660 con 0x7ff01015e5d0 2023-12-12T09:54:28.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.068+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01015cc20 con 0x7ff01015e5d0 2023-12-12T09:54:28.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.069+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff010074450 con 0x7ff01015e5d0 2023-12-12T09:54:28.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.070+0000 7feff77fe640 1 -- 172.21.15.72:0/1530137124 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff008040ce0 con 0x7ff01015e5d0 2023-12-12T09:54:28.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.071+0000 7feff77fe640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feffc06ae40 0x7feffc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.071+0000 7feff77fe640 1 -- 172.21.15.72:0/1530137124 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff0080ac280 con 0x7ff01015e5d0 2023-12-12T09:54:28.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.071+0000 7ff00ed9d640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feffc06ae40 0x7feffc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.072+0000 7ff00ed9d640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feffc06ae40 0x7feffc06d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff0000079a0 tx=0x7ff000008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:28.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.075+0000 7feff77fe640 1 -- 172.21.15.72:0/1530137124 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff008077450 con 0x7ff01015e5d0 2023-12-12T09:54:28.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.278+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff0100adfe0 con 0x7ff01015e5d0 2023-12-12T09:54:28.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.360+0000 7feff77fe640 1 -- 172.21.15.72:0/1530137124 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff00807aa70 con 0x7ff01015e5d0 2023-12-12T09:54:28.363 INFO:teuthology.orchestra.run.smithi072.stderr: removed all blocklist entries 2023-12-12T09:54:28.365 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feffc06ae40 msgr2=0x7feffc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feffc06ae40 0x7feffc06d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff0000079a0 tx=0x7ff000008040 comp rx=0 tx=0).stop 2023-12-12T09:54:28.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff01015e5d0 msgr2=0x7ff010157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff01015e5d0 0x7ff010157210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff008009e80 tx=0x7ff008004a40 comp rx=0 tx=0).stop 2023-12-12T09:54:28.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 shutdown_connections 2023-12-12T09:54:28.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7feffc06ae40 0x7feffc06d300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff01015e5d0 0x7ff010157210 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100ae200 0x7ff01015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 --2- 172.21.15.72:0/1530137124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff010071260 0x7ff01015d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 >> 172.21.15.72:0/1530137124 conn(0x7ff0100b0860 msgr2=0x7ff0100ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:28.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 shutdown_connections 2023-12-12T09:54:28.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.364+0000 7ff014a4a640 1 -- 172.21.15.72:0/1530137124 wait complete. 2023-12-12T09:54:28.386 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-12T09:54:28.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.595+0000 7f9908c9b640 1 Processor -- start 2023-12-12T09:54:28.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.596+0000 7f9908c9b640 1 -- start start 2023-12-12T09:54:28.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.597+0000 7f9908c9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f990406cb00 0x7f99040cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.597+0000 7f9908c9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99040cd980 0x7f990406b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.597+0000 7f9908c9b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990406a5e0 con 0x7f99040ac860 2023-12-12T09:54:28.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.597+0000 7f9908c9b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99040ce1e0 con 0x7f99040cd980 2023-12-12T09:54:28.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.597+0000 7f9908c9b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990406a8a0 con 0x7f990406cb00 2023-12-12T09:54:28.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.597+0000 7f990259c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f99040ac860 0x7f99040acc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35726/0 (socket says 172.21.15.72:35726) 2023-12-12T09:54:28.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.597+0000 7f990259c640 1 -- 172.21.15.72:0/1506149845 learned_addr learned my addr 172.21.15.72:0/1506149845 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:28.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028776440 0 0) 0x7f990406a5e0 con 0x7f99040ac860 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f9902d9d640 1 --2- 172.21.15.72:0/1506149845 >> v2:172.21.15.113:3300/0 conn(0x7f99040cd980 0x7f990406b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f0003520 con 0x7f99040ac860 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f9901d9b640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f990406cb00 0x7f99040cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261357426 0 0) 0x7f98f0003520 con 0x7f99040ac860 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f990406cb00 msgr2=0x7f99040cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f990406cb00 0x7f99040cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 >> v2:172.21.15.113:3300/0 conn(0x7f99040cd980 msgr2=0x7f990406b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 --2- 172.21.15.72:0/1506149845 >> v2:172.21.15.113:3300/0 conn(0x7f99040cd980 0x7f990406b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990406ab80 con 0x7f99040ac860 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f98f40019f0 con 0x7f99040ac860 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.598+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 >> v1:172.21.15.72:6789/0 conn(0x7f99040ac860 legacy=0x7f99040acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f990159a640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0003970 0x7f98f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f990159a640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f990406cb00 0x7f98f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f990159a640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98f0004d00 0x7f98f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040ce1e0 con 0x7f98f0004d00 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990406a8a0 con 0x7f98f0003970 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f000ba00 con 0x7f990406cb00 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f9902d9d640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98f0004d00 0x7f98f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f9901d9b640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f990406cb00 0x7f98f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f990259c640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0003970 0x7f98f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f9902d9d640 1 -- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f990406cb00 msgr2=0x7f98f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f9902d9d640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f990406cb00 0x7f98f00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f9902d9d640 1 -- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0003970 msgr2=0x7f98f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f9902d9d640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f0003970 0x7f98f0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.599+0000 7f9902d9d640 1 -- 172.21.15.72:0/1506149845 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0003520 con 0x7f98f0004d00 2023-12-12T09:54:28.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.600+0000 7f9902d9d640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98f0004d00 0x7f98f0005180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00ba30 tx=0x7f98f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f652fcfbab751ec server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:28.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc032680 con 0x7f98f0004d00 2023-12-12T09:54:28.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc0327e0 con 0x7f98f0004d00 2023-12-12T09:54:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f990159a640 1 -- 172.21.15.72:0/1506149845 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc032b70 con 0x7f98f0004d00 2023-12-12T09:54:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f9908c9b640 1 -- 172.21.15.72:0/1506149845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98f0004d00 msgr2=0x7f98f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f9908c9b640 1 --2- 172.21.15.72:0/1506149845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98f0004d00 0x7f98f0005180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00ba30 tx=0x7f98f00057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f9902d9d640 1 -- 172.21.15.72:0/1506149845 reap_dead start 2023-12-12T09:54:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f9908c9b640 1 -- 172.21.15.72:0/1506149845 shutdown_connections 2023-12-12T09:54:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f9908c9b640 1 -- 172.21.15.72:0/1506149845 >> 172.21.15.72:0/1506149845 conn(0x7f99040cf540 msgr2=0x7f99040cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f9908c9b640 1 -- 172.21.15.72:0/1506149845 shutdown_connections 2023-12-12T09:54:28.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.601+0000 7f9908c9b640 1 -- 172.21.15.72:0/1506149845 wait complete. 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.603+0000 7f9908c9b640 1 Processor -- start 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.603+0000 7f9908c9b640 1 -- start start 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.604+0000 7f9908c9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98f0004d00 0x7f99040c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.604+0000 7f9908c9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990415fad0 0x7f990415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.604+0000 7f9908c9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99040c5360 0x7f990414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.604+0000 7f9908c9b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990415f6b0 con 0x7f990415fad0 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.604+0000 7f9908c9b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990415e790 con 0x7f99040c5360 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.604+0000 7f9908c9b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990415ff30 con 0x7f98f0004d00 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990415fad0 0x7f990415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990415fad0 0x7f990415e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54214/0 (socket says 172.21.15.72:54214) 2023-12-12T09:54:28.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 -- 172.21.15.72:0/2945472299 learned_addr learned my addr 172.21.15.72:0/2945472299 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:28.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f990259c640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98f0004d00 0x7f99040c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 -- 172.21.15.72:0/2945472299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98f0004d00 msgr2=0x7f99040c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98f0004d00 0x7f99040c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 -- 172.21.15.72:0/2945472299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99040c5360 msgr2=0x7f990414eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99040c5360 0x7f990414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 -- 172.21.15.72:0/2945472299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990415ea30 con 0x7f990415fad0 2023-12-12T09:54:28.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f9901d9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990415fad0 0x7f990415e250 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98ec010860 tx=0x7f98ec010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:28.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.605+0000 7f98faffd640 1 -- 172.21.15.72:0/2945472299 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec009610 con 0x7f990415fad0 2023-12-12T09:54:28.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.606+0000 7f98faffd640 1 -- 172.21.15.72:0/2945472299 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec00eae0 con 0x7f990415fad0 2023-12-12T09:54:28.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.606+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990415ed10 con 0x7f990415fad0 2023-12-12T09:54:28.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.606+0000 7f98faffd640 1 -- 172.21.15.72:0/2945472299 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00ee70 con 0x7f990415fad0 2023-12-12T09:54:28.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.606+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990415f2d0 con 0x7f990415fad0 2023-12-12T09:54:28.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.607+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99040745d0 con 0x7f990415fad0 2023-12-12T09:54:28.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.608+0000 7f98faffd640 1 -- 172.21.15.72:0/2945472299 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98ec01c400 con 0x7f990415fad0 2023-12-12T09:54:28.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.609+0000 7f98faffd640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f98e406ae40 0x7f98e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:28.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.609+0000 7f98faffd640 1 -- 172.21.15.72:0/2945472299 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f98ec0899e0 con 0x7f990415fad0 2023-12-12T09:54:28.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.609+0000 7f990259c640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f98e406ae40 0x7f98e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:28.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.610+0000 7f990259c640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f98e406ae40 0x7f98e406d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000f80 tx=0x7f98f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:28.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.614+0000 7f98faffd640 1 -- 172.21.15.72:0/2945472299 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98ec054cb0 con 0x7f990415fad0 2023-12-12T09:54:28.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.873+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f99040b3ec0 con 0x7f990415fad0 2023-12-12T09:54:28.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.874+0000 7f98faffd640 1 -- 172.21.15.72:0/2945472299 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f98ec0582d0 con 0x7f990415fad0 2023-12-12T09:54:28.876 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:28.876 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:54:28.876 INFO:teuthology.orchestra.run.smithi072.stdout: "auth_dump": [ 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mds.a", 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAKLXhlvoC3MRAAoAmTetbcBYxKsaGaD6tAIg==", 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mds.b", 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQALLXhlEGCfFxAAmpAYcdOKIMI4Wg2GyD2V1w==", 2023-12-12T09:54:28.877 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mds.c", 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQALLXhlkWvDBhAAKV9ZAou23rMrzJJ4GURxGw==", 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:54:28.878 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mds.d", 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQALLXhlJcbVKBAAPs/uN737wS83OIndY0JCZQ==", 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.879 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.0", 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAQLXhldS0YHhAANAsQberqgVrbCB5TcgrmsA==", 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.1", 2023-12-12T09:54:28.880 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQATLXhlyVgFKxAA3xZtCVmkuMrqrZCWloqoMA==", 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.2", 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAWLXhlqrpdMxAAmfxGvPquBX/kyiGCi/rjQg==", 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.881 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.3", 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAaLXhl5nXjAhAACmfr78GlROpqJ7UKUwer7A==", 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.4", 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAhLXhlLaouGhAAqURr0qTsHKQby0Th8coLCA==", 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.882 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.5", 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAkLXhlGVwmJRAAX0ujEA/JpOrqD9N9mRR1hg==", 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.883 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.6", 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAnLXhl5VkLLxAAY8lCytX5JEfHFPZ3ABewnw==", 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "osd.7", 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQArLXhlLJaxDBAAGd8VZfX+Ofg80KrvflSEcA==", 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow profile osd", 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile osd", 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.884 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.0", 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQALLXhlQjDrNhAAcu4UiqhTEuide2bwToXy+Q==", 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow r", 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.1", 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAMLXhlMLcyEBAAihZy2JhlSzOc1JbYT37pIw==", 2023-12-12T09:54:28.885 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow r", 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow rw", 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow rwx" 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.2", 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAMLXhlPlybBRAAVk0eZQW2HZk9y6TwvEWIkQ==", 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow r", 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow rw", 2023-12-12T09:54:28.886 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow rwx" 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.3", 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAMLXhlEkM2GxAAvsUAfS00lFnfoviP3gE/mg==", 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow", 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow r", 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow rw", 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow rwx" 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.887 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "client.admin", 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAJLXhlz9ilJBAALwHSd+zMRvGM2rWbTxqGFw==", 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow *", 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr": "allow *", 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow *", 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mgr.x", 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAKLXhlujKiJRAACswJ4xyotHStzLDuXHlGPA==", 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow *", 2023-12-12T09:54:28.888 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile mgr", 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: "entity": "mgr.y", 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "AQAKLXhlmRkdGRAAV7LEG0mSAIVyOfeB2+L1fQ==", 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": { 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": "allow *", 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: "mon": "allow profile mgr", 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "allow *" 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.889 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:28.890 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2023-12-12T09:54:28.890 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.876+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f98e406ae40 msgr2=0x7f98e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.876+0000 7f9908c9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f98e406ae40 0x7f98e406d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000f80 tx=0x7f98f4009040 comp rx=0 tx=0).stop 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990415fad0 msgr2=0x7f990415e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990415fad0 0x7f990415e250 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98ec010860 tx=0x7f98ec010d20 comp rx=0 tx=0).stop 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 shutdown_connections 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f98e406ae40 0x7f98e406d300 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99040c5360 0x7f990414eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990415fad0 0x7f990415e250 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 --2- 172.21.15.72:0/2945472299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98f0004d00 0x7f99040c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 >> 172.21.15.72:0/2945472299 conn(0x7f99040cf540 msgr2=0x7f990406cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:28.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 shutdown_connections 2023-12-12T09:54:28.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:28.877+0000 7f9908c9b640 1 -- 172.21.15.72:0/2945472299 wait complete. 2023-12-12T09:54:28.899 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:29.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.109+0000 7fdd9a448640 1 Processor -- start 2023-12-12T09:54:29.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.109+0000 7fdd9a448640 1 -- start start 2023-12-12T09:54:29.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.110+0000 7fdd9a448640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94072180 0x7fdd940b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.111+0000 7fdd9a448640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940b3bb0 con 0x7fdd9406c410 2023-12-12T09:54:29.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.111+0000 7fdd9a448640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd941441e0 con 0x7fdd94143e00 2023-12-12T09:54:29.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.111+0000 7fdd9a448640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd94072680 con 0x7fdd94072180 2023-12-12T09:54:29.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.111+0000 7fdd989e5640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fdd9406c410 0x7fdd9415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35740/0 (socket says 172.21.15.72:35740) 2023-12-12T09:54:29.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.111+0000 7fdd989e5640 1 -- 172.21.15.72:0/1660556103 learned_addr learned my addr 172.21.15.72:0/1660556103 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.111+0000 7fdd8bfff640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94072180 0x7fdd940b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.111+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342377327 0 0) 0x7fdd940b3bb0 con 0x7fdd9406c410 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.111+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd80003520 con 0x7fdd9406c410 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357137415 0 0) 0x7fdd80003520 con 0x7fdd9406c410 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94072180 msgr2=0x7fdd940b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94072180 0x7fdd940b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 >> v1:172.21.15.113:6789/0 conn(0x7fdd94143e00 legacy=0x7fdd940b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940b1e30 con 0x7fdd9406c410 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fdd8c0019f0 con 0x7fdd9406c410 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 >> v1:172.21.15.72:6789/0 conn(0x7fdd9406c410 legacy=0x7fdd9415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd80003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94072180 0x7fdd800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80004d00 0x7fdd80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd94072680 con 0x7fdd800039a0 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8000ba30 con 0x7fdd80004d00 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8000bba0 con 0x7fdd94072180 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.112+0000 7fdd93fff640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd80003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.113+0000 7fdd8bfff640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94072180 0x7fdd800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.113+0000 7fdd93fff640 1 -- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94072180 msgr2=0x7fdd800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.113+0000 7fdd93fff640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94072180 0x7fdd800047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.113+0000 7fdd93fff640 1 -- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80004d00 msgr2=0x7fdd80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.113+0000 7fdd93fff640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80004d00 0x7fdd80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.113+0000 7fdd93fff640 1 -- 172.21.15.72:0/1660556103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80003520 con 0x7fdd800039a0 2023-12-12T09:54:29.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.114+0000 7fdd93fff640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd80003d80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdd84001860 tx=0x7fdd800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacacbd7c4f1413b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.114+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84033630 con 0x7fdd800039a0 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.114+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd840337b0 con 0x7fdd800039a0 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.114+0000 7fdd937fe640 1 -- 172.21.15.72:0/1660556103 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84033b60 con 0x7fdd800039a0 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.115+0000 7fdd9a448640 1 -- 172.21.15.72:0/1660556103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 msgr2=0x7fdd80003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.115+0000 7fdd9a448640 1 --2- 172.21.15.72:0/1660556103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd80003d80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdd84001860 tx=0x7fdd800057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.115+0000 7fdd989e5640 1 -- 172.21.15.72:0/1660556103 reap_dead start 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.115+0000 7fdd9a448640 1 -- 172.21.15.72:0/1660556103 shutdown_connections 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.115+0000 7fdd9a448640 1 -- 172.21.15.72:0/1660556103 >> 172.21.15.72:0/1660556103 conn(0x7fdd940b0a90 msgr2=0x7fdd940b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.115+0000 7fdd9a448640 1 -- 172.21.15.72:0/1660556103 shutdown_connections 2023-12-12T09:54:29.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.115+0000 7fdd9a448640 1 -- 172.21.15.72:0/1660556103 wait complete. 2023-12-12T09:54:29.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.117+0000 7fdd9a448640 1 Processor -- start 2023-12-12T09:54:29.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.117+0000 7fdd9a448640 1 -- start start 2023-12-12T09:54:29.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd9a448640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd940ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd9a448640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94140750 0x7fdd9415a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd9a448640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd94141110 0x7fdd9413d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd9a448640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940ce0c0 con 0x7fdd800039a0 2023-12-12T09:54:29.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd9a448640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940cd4b0 con 0x7fdd94141110 2023-12-12T09:54:29.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd9a448640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940cd790 con 0x7fdd94140750 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd940ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd940ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54236/0 (socket says 172.21.15.72:54236) 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd93fff640 1 -- 172.21.15.72:0/3595296856 learned_addr learned my addr 172.21.15.72:0/3595296856 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd93fff640 1 -- 172.21.15.72:0/3595296856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94140750 msgr2=0x7fdd9415a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd93fff640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94140750 0x7fdd9415a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd93fff640 1 -- 172.21.15.72:0/3595296856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd94141110 msgr2=0x7fdd9413d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd93fff640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd94141110 0x7fdd9413d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.118+0000 7fdd93fff640 1 -- 172.21.15.72:0/3595296856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940cda70 con 0x7fdd800039a0 2023-12-12T09:54:29.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.119+0000 7fdd93fff640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd940ce550 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdd84008fd0 tx=0x7fdd84001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:29.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.119+0000 7fdd917fa640 1 -- 172.21.15.72:0/3595296856 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84007050 con 0x7fdd800039a0 2023-12-12T09:54:29.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.119+0000 7fdd917fa640 1 -- 172.21.15.72:0/3595296856 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd840071b0 con 0x7fdd800039a0 2023-12-12T09:54:29.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.119+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd940cdd50 con 0x7fdd800039a0 2023-12-12T09:54:29.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.119+0000 7fdd917fa640 1 -- 172.21.15.72:0/3595296856 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84007540 con 0x7fdd800039a0 2023-12-12T09:54:29.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.119+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd941415a0 con 0x7fdd800039a0 2023-12-12T09:54:29.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.120+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94141880 con 0x7fdd800039a0 2023-12-12T09:54:29.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.121+0000 7fdd917fa640 1 -- 172.21.15.72:0/3595296856 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd840067d0 con 0x7fdd800039a0 2023-12-12T09:54:29.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.122+0000 7fdd917fa640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd7406ae40 0x7fdd7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.122+0000 7fdd917fa640 1 -- 172.21.15.72:0/3595296856 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdd840ab210 con 0x7fdd800039a0 2023-12-12T09:54:29.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.122+0000 7fdd8bfff640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd7406ae40 0x7fdd7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.123+0000 7fdd8bfff640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd7406ae40 0x7fdd7406d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd940ae6f0 tx=0x7fdd7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:29.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.126+0000 7fdd917fa640 1 -- 172.21.15.72:0/3595296856 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd84076430 con 0x7fdd800039a0 2023-12-12T09:54:29.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.319+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd940745d0 con 0x7fdd800039a0 2023-12-12T09:54:29.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.321+0000 7fdd917fa640 1 -- 172.21.15.72:0/3595296856 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13062 (secure 0 0 0) 0x7fdd84079a50 con 0x7fdd800039a0 2023-12-12T09:54:29.323 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:29.323 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":47,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:28.378989+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:29.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd7406ae40 msgr2=0x7fdd7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd7406ae40 0x7fdd7406d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd940ae6f0 tx=0x7fdd7c008040 comp rx=0 tx=0).stop 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 msgr2=0x7fdd940ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd940ce550 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdd84008fd0 tx=0x7fdd84001230 comp rx=0 tx=0).stop 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 shutdown_connections 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdd7406ae40 0x7fdd7406d300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd94141110 0x7fdd9413d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd94140750 0x7fdd9415a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 --2- 172.21.15.72:0/3595296856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd800039a0 0x7fdd940ce550 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.323+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 >> 172.21.15.72:0/3595296856 conn(0x7fdd940b0a90 msgr2=0x7fdd9406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.324+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 shutdown_connections 2023-12-12T09:54:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.324+0000 7fdd9a448640 1 -- 172.21.15.72:0/3595296856 wait complete. 2023-12-12T09:54:29.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:54:29.345 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:54:29.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.558+0000 7f13f9ec9640 1 Processor -- start 2023-12-12T09:54:29.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.559+0000 7f13f9ec9640 1 -- start start 2023-12-12T09:54:29.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.560+0000 7f13f9ec9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f13f415d0e0 0x7f13f40ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.560+0000 7f13f9ec9640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f13f4072210 0x7f13f4161530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.560+0000 7f13f9ec9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f4160dc0 con 0x7f13f4161a70 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.560+0000 7f13f9ec9640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f4070280 con 0x7f13f4072210 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.560+0000 7f13f9ec9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f4070560 con 0x7f13f415d0e0 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.561+0000 7f13f2ffd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f13f4072210 0x7f13f4161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.561+0000 7f13f2ffd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f13f4072210 0x7f13f4161530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54252/0 (socket says 172.21.15.72:54252) 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.561+0000 7f13f2ffd640 1 -- 172.21.15.72:0/1147689145 learned_addr learned my addr 172.21.15.72:0/1147689145 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.561+0000 7f13f37fe640 1 --2- 172.21.15.72:0/1147689145 >> v2:172.21.15.113:3300/0 conn(0x7f13f415d0e0 0x7f13f40ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.561+0000 7f13f2ffd640 1 -- 172.21.15.72:0/1147689145 >> v2:172.21.15.113:3300/0 conn(0x7f13f415d0e0 msgr2=0x7f13f40ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:29.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.561+0000 7f13f2ffd640 1 --2- 172.21.15.72:0/1147689145 >> v2:172.21.15.113:3300/0 conn(0x7f13f415d0e0 0x7f13f40ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.561+0000 7f13f2ffd640 1 -- 172.21.15.72:0/1147689145 >> v1:172.21.15.113:6789/0 conn(0x7f13f4161a70 legacy=0x7f13f40c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:29.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.561+0000 7f13f2ffd640 1 -- 172.21.15.72:0/1147689145 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f4070840 con 0x7f13f4072210 2023-12-12T09:54:29.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.562+0000 7f13f2ffd640 1 --2- 172.21.15.72:0/1147689145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f4072210 0x7f13f4161530 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f13f40b4e20 tx=0x7f13e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b640bb17ce99055 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:29.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f27fc640 1 -- 172.21.15.72:0/1147689145 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8030860 con 0x7f13f4072210 2023-12-12T09:54:29.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f27fc640 1 -- 172.21.15.72:0/1147689145 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e80309c0 con 0x7f13f4072210 2023-12-12T09:54:29.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f27fc640 1 -- 172.21.15.72:0/1147689145 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8030d50 con 0x7f13f4072210 2023-12-12T09:54:29.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f9ec9640 1 -- 172.21.15.72:0/1147689145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f4072210 msgr2=0x7f13f4161530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/1147689145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f4072210 0x7f13f4161530 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f13f40b4e20 tx=0x7f13e802b9a0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f9ec9640 1 -- 172.21.15.72:0/1147689145 shutdown_connections 2023-12-12T09:54:29.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/1147689145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f4072210 0x7f13f4161530 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/1147689145 >> v2:172.21.15.113:3300/0 conn(0x7f13f415d0e0 0x7f13f40ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.563+0000 7f13f9ec9640 1 -- 172.21.15.72:0/1147689145 >> 172.21.15.72:0/1147689145 conn(0x7f13f40cee70 msgr2=0x7f13f40cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:29.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.564+0000 7f13f9ec9640 1 -- 172.21.15.72:0/1147689145 shutdown_connections 2023-12-12T09:54:29.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.564+0000 7f13f9ec9640 1 -- 172.21.15.72:0/1147689145 wait complete. 2023-12-12T09:54:29.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.565+0000 7f13f9ec9640 1 Processor -- start 2023-12-12T09:54:29.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.565+0000 7f13f9ec9640 1 -- start start 2023-12-12T09:54:29.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.566+0000 7f13f9ec9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13f4072210 0x7f13f40c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.566+0000 7f13f9ec9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f415d0e0 0x7f13f4145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.566+0000 7f13f9ec9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13f4146140 0x7f13f4147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.566+0000 7f13f9ec9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f4145960 con 0x7f13f415d0e0 2023-12-12T09:54:29.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.566+0000 7f13f9ec9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f4147290 con 0x7f13f4146140 2023-12-12T09:54:29.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.566+0000 7f13f9ec9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f4146680 con 0x7f13f4072210 2023-12-12T09:54:29.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f415d0e0 0x7f13f4145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f415d0e0 0x7f13f4145c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54258/0 (socket says 172.21.15.72:54258) 2023-12-12T09:54:29.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 -- 172.21.15.72:0/2978601296 learned_addr learned my addr 172.21.15.72:0/2978601296 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:29.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 -- 172.21.15.72:0/2978601296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13f4072210 msgr2=0x7f13f40c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13f4072210 0x7f13f40c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 -- 172.21.15.72:0/2978601296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13f4146140 msgr2=0x7f13f4147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f3fff640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13f4146140 0x7f13f4147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13f4146140 0x7f13f4147c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 -- 172.21.15.72:0/2978601296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80094d0 con 0x7f13f415d0e0 2023-12-12T09:54:29.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.567+0000 7f13f2ffd640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f415d0e0 0x7f13f4145c00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f13e802b9e0 tx=0x7f13e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:29.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.568+0000 7f13dbfff640 1 -- 172.21.15.72:0/2978601296 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8039d60 con 0x7f13f415d0e0 2023-12-12T09:54:29.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.568+0000 7f13dbfff640 1 -- 172.21.15.72:0/2978601296 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e8043400 con 0x7f13f415d0e0 2023-12-12T09:54:29.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.568+0000 7f13dbfff640 1 -- 172.21.15.72:0/2978601296 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e80437b0 con 0x7f13f415d0e0 2023-12-12T09:54:29.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.568+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f4146960 con 0x7f13f415d0e0 2023-12-12T09:54:29.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.568+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f4146f20 con 0x7f13f415d0e0 2023-12-12T09:54:29.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.569+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f41470b0 con 0x7f13f415d0e0 2023-12-12T09:54:29.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.570+0000 7f13dbfff640 1 -- 172.21.15.72:0/2978601296 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13e8043910 con 0x7f13f415d0e0 2023-12-12T09:54:29.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.571+0000 7f13dbfff640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f13e006ae40 0x7f13e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.571+0000 7f13dbfff640 1 -- 172.21.15.72:0/2978601296 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f13e803e070 con 0x7f13f415d0e0 2023-12-12T09:54:29.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.571+0000 7f13f37fe640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f13e006ae40 0x7f13e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.572+0000 7f13f37fe640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f13e006ae40 0x7f13e006d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f13e40070b0 tx=0x7f13e4007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:29.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.575+0000 7f13dbfff640 1 -- 172.21.15.72:0/2978601296 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13e8077b50 con 0x7f13f415d0e0 2023-12-12T09:54:29.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.764+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13f4074450 con 0x7f13f415d0e0 2023-12-12T09:54:29.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.765+0000 7f13dbfff640 1 -- 172.21.15.72:0/2978601296 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13e807b170 con 0x7f13f415d0e0 2023-12-12T09:54:29.767 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:54:29.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f13e006ae40 msgr2=0x7f13e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f13e006ae40 0x7f13e006d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f13e40070b0 tx=0x7f13e4007040 comp rx=0 tx=0).stop 2023-12-12T09:54:29.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f415d0e0 msgr2=0x7f13f4145c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f415d0e0 0x7f13f4145c00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f13e802b9e0 tx=0x7f13e8004730 comp rx=0 tx=0).stop 2023-12-12T09:54:29.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 shutdown_connections 2023-12-12T09:54:29.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f13e006ae40 0x7f13e006d300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13f4146140 0x7f13f4147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13f415d0e0 0x7f13f4145c00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 --2- 172.21.15.72:0/2978601296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13f4072210 0x7f13f40c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 >> 172.21.15.72:0/2978601296 conn(0x7f13f40cee70 msgr2=0x7f13f40cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:29.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 shutdown_connections 2023-12-12T09:54:29.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.768+0000 7f13f9ec9640 1 -- 172.21.15.72:0/2978601296 wait complete. 2023-12-12T09:54:29.787 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-12T09:54:29.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.991+0000 7fc9e2139640 1 Processor -- start 2023-12-12T09:54:29.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.991+0000 7fc9e2139640 1 -- start start 2023-12-12T09:54:29.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.993+0000 7fc9e2139640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc06c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.993+0000 7fc9e2139640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9dc06de20 0x7fc9dc15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.993+0000 7fc9e2139640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc9dc15e0e0 0x7fc9dc15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.993+0000 7fc9e2139640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9dc15e550 con 0x7fc9dc15e0e0 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.993+0000 7fc9e2139640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9dc06ca40 con 0x7fc9dc06de20 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.993+0000 7fc9e2139640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc071cd0 con 0x7fc9dc157a60 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.993+0000 7fc9dbfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc9dc15e0e0 0x7fc9dc15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9dbfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc9dc15e0e0 0x7fc9dc15a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54266/0 (socket says 172.21.15.72:54266) 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9dbfff640 1 -- 172.21.15.72:0/997940929 learned_addr learned my addr 172.21.15.72:0/997940929 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9db7fe640 1 --2- 172.21.15.72:0/997940929 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9d3fff640 1 --2- 172.21.15.72:0/997940929 >> v2:172.21.15.113:3300/0 conn(0x7fc9dc06de20 0x7fc9dc15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9dbfff640 1 -- 172.21.15.72:0/997940929 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 msgr2=0x7fc9dc06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9dbfff640 1 --2- 172.21.15.72:0/997940929 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc06c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9dbfff640 1 -- 172.21.15.72:0/997940929 >> v2:172.21.15.113:3300/0 conn(0x7fc9dc06de20 msgr2=0x7fc9dc15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9dbfff640 1 --2- 172.21.15.72:0/997940929 >> v2:172.21.15.113:3300/0 conn(0x7fc9dc06de20 0x7fc9dc15dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9dbfff640 1 -- 172.21.15.72:0/997940929 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc070f10 con 0x7fc9dc15e0e0 2023-12-12T09:54:29.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.994+0000 7fc9db7fe640 1 --2- 172.21.15.72:0/997940929 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc06c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.995+0000 7fc9dbfff640 1 --2- 172.21.15.72:0/997940929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc15a5b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc06b810 tx=0x7fc9d402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcffa04570fe72a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.995+0000 7fc9daffd640 1 -- 172.21.15.72:0/997940929 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d402c9f0 con 0x7fc9dc15e0e0 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9daffd640 1 -- 172.21.15.72:0/997940929 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d402cb50 con 0x7fc9dc15e0e0 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9daffd640 1 -- 172.21.15.72:0/997940929 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40343f0 con 0x7fc9dc15e0e0 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 -- 172.21.15.72:0/997940929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 msgr2=0x7fc9dc15a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 --2- 172.21.15.72:0/997940929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc15a5b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc06b810 tx=0x7fc9d402be50 comp rx=0 tx=0).stop 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 -- 172.21.15.72:0/997940929 shutdown_connections 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 --2- 172.21.15.72:0/997940929 >> v2:172.21.15.113:3300/0 conn(0x7fc9dc06de20 0x7fc9dc15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 --2- 172.21.15.72:0/997940929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc15a5b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 --2- 172.21.15.72:0/997940929 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc06c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 -- 172.21.15.72:0/997940929 >> 172.21.15.72:0/997940929 conn(0x7fc9dc0b0a90 msgr2=0x7fc9dc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:29.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 -- 172.21.15.72:0/997940929 shutdown_connections 2023-12-12T09:54:30.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.996+0000 7fc9e2139640 1 -- 172.21.15.72:0/997940929 wait complete. 2023-12-12T09:54:30.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.998+0000 7fc9e2139640 1 Processor -- start 2023-12-12T09:54:30.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.998+0000 7fc9e2139640 1 -- start start 2023-12-12T09:54:30.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.999+0000 7fc9e2139640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9dc06de20 0x7fc9dc15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.999+0000 7fc9e2139640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.999+0000 7fc9e2139640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.999+0000 7fc9e2139640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc148190 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.999+0000 7fc9e2139640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc148430 con 0x7fc9dc06de20 2023-12-12T09:54:30.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:29.999+0000 7fc9e2139640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc148710 con 0x7fc9dc157a60 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9dbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54272/0 (socket says 172.21.15.72:54272) 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9dbfff640 1 -- 172.21.15.72:0/2981153727 learned_addr learned my addr 172.21.15.72:0/2981153727 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9db7fe640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9dc06de20 0x7fc9dc15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9d3fff640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9dbfff640 1 -- 172.21.15.72:0/2981153727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 msgr2=0x7fc9dc155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9dbfff640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9dbfff640 1 -- 172.21.15.72:0/2981153727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9dc06de20 msgr2=0x7fc9dc15bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9dbfff640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9dc06de20 0x7fc9dc15bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.000+0000 7fc9dbfff640 1 -- 172.21.15.72:0/2981153727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4009b30 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.001+0000 7fc9dbfff640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc155e50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4036040 tx=0x7fc9d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:30.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.001+0000 7fc9d8ff9640 1 -- 172.21.15.72:0/2981153727 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4009510 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.001+0000 7fc9d8ff9640 1 -- 172.21.15.72:0/2981153727 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d403e450 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.001+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc1489f0 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.001+0000 7fc9d8ff9640 1 -- 172.21.15.72:0/2981153727 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d403e800 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.001+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc15dd20 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.002+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc148f10 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.003+0000 7fc9d8ff9640 1 -- 172.21.15.72:0/2981153727 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9d403e960 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.004+0000 7fc9d8ff9640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc9bc06ae40 0x7fc9bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.004+0000 7fc9d8ff9640 1 -- 172.21.15.72:0/2981153727 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc9d40ac1e0 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.005+0000 7fc9db7fe640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc9bc06ae40 0x7fc9bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.005+0000 7fc9db7fe640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc9bc06ae40 0x7fc9bc06d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0ce1e0 tx=0x7fc9cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:30.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.009+0000 7fc9d8ff9640 1 -- 172.21.15.72:0/2981153727 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9dc148f10 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.199+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc9dc148f10 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.200+0000 7fc9d8ff9640 1 -- 172.21.15.72:0/2981153727 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc9dc148f10 con 0x7fc9dc15e0e0 2023-12-12T09:54:30.202 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2023-12-12T09:54:30.202 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-12-12T09:54:30.202 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 1, 2023-12-12T09:54:30.202 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": ".mgr" 2023-12-12T09:54:30.202 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-12-12T09:54:30.202 INFO:teuthology.orchestra.run.smithi072.stdout:] 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.202+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc9bc06ae40 msgr2=0x7fc9bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.202+0000 7fc9e2139640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc9bc06ae40 0x7fc9bc06d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0ce1e0 tx=0x7fc9cc008040 comp rx=0 tx=0).stop 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.202+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 msgr2=0x7fc9dc155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.202+0000 7fc9e2139640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc155e50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4036040 tx=0x7fc9d4004a40 comp rx=0 tx=0).stop 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.203+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 shutdown_connections 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.203+0000 7fc9e2139640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fc9bc06ae40 0x7fc9bc06d300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.203+0000 7fc9e2139640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9dc15e0e0 0x7fc9dc155e50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.203+0000 7fc9e2139640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9dc157a60 0x7fc9dc155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.203+0000 7fc9e2139640 1 --2- 172.21.15.72:0/2981153727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9dc06de20 0x7fc9dc15bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.203+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 >> 172.21.15.72:0/2981153727 conn(0x7fc9dc0b0a90 msgr2=0x7fc9dc0ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:30.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.203+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 shutdown_connections 2023-12-12T09:54:30.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.203+0000 7fc9e2139640 1 -- 172.21.15.72:0/2981153727 wait complete. 2023-12-12T09:54:30.223 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-12T09:54:30.223 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-12T09:54:30.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.429+0000 7f56c3c10640 1 Processor -- start 2023-12-12T09:54:30.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.429+0000 7f56c3c10640 1 -- start start 2023-12-12T09:54:30.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.431+0000 7f56c3c10640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 0x7f56bc06c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.431+0000 7f56c3c10640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56bc15ec10 0x7f56bc15f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.431+0000 7f56c3c10640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc15f5a0 con 0x7f56bc0724a0 2023-12-12T09:54:30.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.431+0000 7f56c3c10640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56bc0b3c50 con 0x7f56bc15ec10 2023-12-12T09:54:30.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.431+0000 7f56c3c10640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b3960 con 0x7f56bc1448a0 2023-12-12T09:54:30.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.431+0000 7f56c11ab640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f56bc0724a0 0x7f56bc0b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35756/0 (socket says 172.21.15.72:35756) 2023-12-12T09:54:30.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.431+0000 7f56c11ab640 1 -- 172.21.15.72:0/892224997 learned_addr learned my addr 172.21.15.72:0/892224997 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:30.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c19ac640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 0x7f56bc06c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c21ad640 1 --2- 172.21.15.72:0/892224997 >> v2:172.21.15.113:3300/0 conn(0x7f56bc15ec10 0x7f56bc15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922182847 0 0) 0x7f56b0001110 con 0x7f56bc0724a0 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a4003520 con 0x7f56bc0724a0 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2545025724 0 0) 0x7f56a4003520 con 0x7f56bc0724a0 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 msgr2=0x7f56bc06c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 0x7f56bc06c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 >> v2:172.21.15.113:3300/0 conn(0x7f56bc15ec10 msgr2=0x7f56bc15f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 --2- 172.21.15.72:0/892224997 >> v2:172.21.15.113:3300/0 conn(0x7f56bc15ec10 0x7f56bc15f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc071140 con 0x7f56bc0724a0 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f56b0001800 con 0x7f56bc0724a0 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 >> v1:172.21.15.72:6789/0 conn(0x7f56bc0724a0 legacy=0x7f56bc0b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4003970 0x7f56a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 0x7f56a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b3c50 con 0x7f56a4004d00 2023-12-12T09:54:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b3960 con 0x7f56a4003970 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.432+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a400b980 con 0x7f56bc1448a0 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c21ad640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c19ac640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4003970 0x7f56a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c11ab640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 0x7f56a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c21ad640 1 -- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 msgr2=0x7f56a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c21ad640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 0x7f56a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c21ad640 1 -- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4003970 msgr2=0x7f56a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c21ad640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4003970 0x7f56a4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c21ad640 1 -- 172.21.15.72:0/892224997 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4003520 con 0x7f56a4004d00 2023-12-12T09:54:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.433+0000 7f56c11ab640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc1448a0 0x7f56a40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:30.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.434+0000 7f56c21ad640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f56b400ba60 tx=0x7f56a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c1067b90207df2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:30.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.434+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4032660 con 0x7f56a4004d00 2023-12-12T09:54:30.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.434+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b40327c0 con 0x7f56a4004d00 2023-12-12T09:54:30.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.434+0000 7f56c09aa640 1 -- 172.21.15.72:0/892224997 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4032b50 con 0x7f56a4004d00 2023-12-12T09:54:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.435+0000 7f56c3c10640 1 -- 172.21.15.72:0/892224997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56a4004d00 msgr2=0x7f56a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.435+0000 7f56c3c10640 1 --2- 172.21.15.72:0/892224997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f56b400ba60 tx=0x7f56a40057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.435+0000 7f56c21ad640 1 -- 172.21.15.72:0/892224997 reap_dead start 2023-12-12T09:54:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.435+0000 7f56c3c10640 1 -- 172.21.15.72:0/892224997 shutdown_connections 2023-12-12T09:54:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.435+0000 7f56c3c10640 1 -- 172.21.15.72:0/892224997 >> 172.21.15.72:0/892224997 conn(0x7f56bc0b0a90 msgr2=0x7f56bc0b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.435+0000 7f56c3c10640 1 -- 172.21.15.72:0/892224997 shutdown_connections 2023-12-12T09:54:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.435+0000 7f56c3c10640 1 -- 172.21.15.72:0/892224997 wait complete. 2023-12-12T09:54:30.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.437+0000 7f56c3c10640 1 Processor -- start 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.437+0000 7f56c3c10640 1 -- start start 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c3c10640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4004d00 0x7f56bc13d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c3c10640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56bc13f680 0x7f56bc070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c3c10640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc13fb00 0x7f56bc070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c3c10640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc150290 con 0x7f56bc13f680 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c3c10640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc13d560 con 0x7f56a4004d00 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c3c10640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc14d040 con 0x7f56bc13fb00 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c11ab640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56bc13f680 0x7f56bc070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c11ab640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56bc13f680 0x7f56bc070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54282/0 (socket says 172.21.15.72:54282) 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.438+0000 7f56c11ab640 1 -- 172.21.15.72:0/194110785 learned_addr learned my addr 172.21.15.72:0/194110785 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c19ac640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4004d00 0x7f56bc13d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c21ad640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc13fb00 0x7f56bc070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c11ab640 1 -- 172.21.15.72:0/194110785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc13fb00 msgr2=0x7f56bc070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c11ab640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc13fb00 0x7f56bc070950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c11ab640 1 -- 172.21.15.72:0/194110785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4004d00 msgr2=0x7f56bc13d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:30.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c11ab640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4004d00 0x7f56bc13d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:30.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c11ab640 1 -- 172.21.15.72:0/194110785 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc14d320 con 0x7f56bc13f680 2023-12-12T09:54:30.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c11ab640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56bc13f680 0x7f56bc070410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f56b000dae0 tx=0x7f56b0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56ba7fc640 1 -- 172.21.15.72:0/194110785 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b00075a0 con 0x7f56bc13f680 2023-12-12T09:54:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56ba7fc640 1 -- 172.21.15.72:0/194110785 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b0007700 con 0x7f56bc13f680 2023-12-12T09:54:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56ba7fc640 1 -- 172.21.15.72:0/194110785 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b00049e0 con 0x7f56bc13f680 2023-12-12T09:54:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc13eba0 con 0x7f56bc13f680 2023-12-12T09:54:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.439+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc13f160 con 0x7f56bc13f680 2023-12-12T09:54:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.440+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc13ea00 con 0x7f56bc13f680 2023-12-12T09:54:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.441+0000 7f56ba7fc640 1 -- 172.21.15.72:0/194110785 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56b0004b40 con 0x7f56bc13f680 2023-12-12T09:54:30.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.442+0000 7f56ba7fc640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f56a806ae40 0x7f56a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:30.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.442+0000 7f56ba7fc640 1 -- 172.21.15.72:0/194110785 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f56b0088e20 con 0x7f56bc13f680 2023-12-12T09:54:30.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.442+0000 7f56c19ac640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f56a806ae40 0x7f56a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:30.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.443+0000 7f56c19ac640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f56a806ae40 0x7f56a806d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009fc0 tx=0x7f56ac00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:30.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.447+0000 7f56ba7fc640 1 -- 172.21.15.72:0/194110785 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56bc13ea00 con 0x7f56bc13f680 2023-12-12T09:54:30.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:30.640+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f56bc15f5a0 con 0x7f56bc13f680 2023-12-12T09:54:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.391+0000 7f56ba7fc640 1 -- 172.21.15.72:0/194110785 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f56bc15f5a0 con 0x7f56bc13f680 2023-12-12T09:54:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' created 2023-12-12T09:54:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.396+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f56a806ae40 msgr2=0x7f56a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.396+0000 7f56c3c10640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f56a806ae40 0x7f56a806d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009fc0 tx=0x7f56ac00b040 comp rx=0 tx=0).stop 2023-12-12T09:54:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.396+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56bc13f680 msgr2=0x7f56bc070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.396+0000 7f56c3c10640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56bc13f680 0x7f56bc070410 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f56b000dae0 tx=0x7f56b0007e10 comp rx=0 tx=0).stop 2023-12-12T09:54:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.397+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 shutdown_connections 2023-12-12T09:54:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.397+0000 7f56c3c10640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f56a806ae40 0x7f56a806d300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.397+0000 7f56c3c10640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56bc13fb00 0x7f56bc070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.397+0000 7f56c3c10640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f56bc13f680 0x7f56bc070410 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.397+0000 7f56c3c10640 1 --2- 172.21.15.72:0/194110785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56a4004d00 0x7f56bc13d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.397+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 >> 172.21.15.72:0/194110785 conn(0x7f56bc0b0a90 msgr2=0x7f56bc142cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.397+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 shutdown_connections 2023-12-12T09:54:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.397+0000 7f56c3c10640 1 -- 172.21.15.72:0/194110785 wait complete. 2023-12-12T09:54:31.423 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-12T09:54:31.627 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.625+0000 7f0712f3c640 1 Processor -- start 2023-12-12T09:54:31.627 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.625+0000 7f0712f3c640 1 -- start start 2023-12-12T09:54:31.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.627+0000 7f0712f3c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f070c155890 0x7f070c15f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.627+0000 7f0712f3c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f070c15f9b0 0x7f070c0cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.627+0000 7f0712f3c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c15ece0 con 0x7f070c1554b0 2023-12-12T09:54:31.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.627+0000 7f0712f3c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c15bd60 con 0x7f070c155890 2023-12-12T09:54:31.629 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.627+0000 7f0712f3c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c1416c0 con 0x7f070c15f9b0 2023-12-12T09:54:31.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.627+0000 7f070bfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f070c155890 0x7f070c15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.628+0000 7f070bfff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f070c155890 0x7f070c15f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54284/0 (socket says 172.21.15.72:54284) 2023-12-12T09:54:31.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.628+0000 7f070bfff640 1 -- 172.21.15.72:0/1016938487 learned_addr learned my addr 172.21.15.72:0/1016938487 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:31.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.628+0000 7f07114d9640 1 --2- 172.21.15.72:0/1016938487 >> v2:172.21.15.113:3300/0 conn(0x7f070c15f9b0 0x7f070c0cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.628+0000 7f070bfff640 1 -- 172.21.15.72:0/1016938487 >> v2:172.21.15.113:3300/0 conn(0x7f070c15f9b0 msgr2=0x7f070c0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:31.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.628+0000 7f070bfff640 1 --2- 172.21.15.72:0/1016938487 >> v2:172.21.15.113:3300/0 conn(0x7f070c15f9b0 0x7f070c0cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.628+0000 7f070bfff640 1 -- 172.21.15.72:0/1016938487 >> v1:172.21.15.113:6789/0 conn(0x7f070c1554b0 legacy=0x7f070c15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:31.630 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.628+0000 7f070bfff640 1 -- 172.21.15.72:0/1016938487 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c15bf80 con 0x7f070c155890 2023-12-12T09:54:31.631 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.629+0000 7f070bfff640 1 --2- 172.21.15.72:0/1016938487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c155890 0x7f070c15f470 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f070c0cceb0 tx=0x7f06f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daadc68122d316bc server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:31.631 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.629+0000 7f070b7fe640 1 -- 172.21.15.72:0/1016938487 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f80317e0 con 0x7f070c155890 2023-12-12T09:54:31.631 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f070b7fe640 1 -- 172.21.15.72:0/1016938487 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f8031940 con 0x7f070c155890 2023-12-12T09:54:31.631 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f070b7fe640 1 -- 172.21.15.72:0/1016938487 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8031cd0 con 0x7f070c155890 2023-12-12T09:54:31.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f0712f3c640 1 -- 172.21.15.72:0/1016938487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c155890 msgr2=0x7f070c15f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:31.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1016938487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c155890 0x7f070c15f470 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f070c0cceb0 tx=0x7f06f802b880 comp rx=0 tx=0).stop 2023-12-12T09:54:31.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f0712f3c640 1 -- 172.21.15.72:0/1016938487 shutdown_connections 2023-12-12T09:54:31.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1016938487 >> v2:172.21.15.113:3300/0 conn(0x7f070c15f9b0 0x7f070c0cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1016938487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c155890 0x7f070c15f470 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.633 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f0712f3c640 1 -- 172.21.15.72:0/1016938487 >> 172.21.15.72:0/1016938487 conn(0x7f070c06a880 msgr2=0x7f070c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:31.633 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f0712f3c640 1 -- 172.21.15.72:0/1016938487 shutdown_connections 2023-12-12T09:54:31.633 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.630+0000 7f0712f3c640 1 -- 172.21.15.72:0/1016938487 wait complete. 2023-12-12T09:54:31.634 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.632+0000 7f0712f3c640 1 Processor -- start 2023-12-12T09:54:31.634 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.632+0000 7f0712f3c640 1 -- start start 2023-12-12T09:54:31.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.633+0000 7f0712f3c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c1554b0 0x7f070c143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.633+0000 7f0712f3c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c155890 0x7f070c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.633+0000 7f0712f3c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f070c15f9b0 0x7f070c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.633+0000 7f0712f3c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c13d180 con 0x7f070c1554b0 2023-12-12T09:54:31.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.633+0000 7f0712f3c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c15c6c0 con 0x7f070c155890 2023-12-12T09:54:31.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.633+0000 7f0712f3c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c15cf60 con 0x7f070c15f9b0 2023-12-12T09:54:31.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.633+0000 7f0710cd8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c1554b0 0x7f070c143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.635 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.633+0000 7f0710cd8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c1554b0 0x7f070c143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54288/0 (socket says 172.21.15.72:54288) 2023-12-12T09:54:31.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f0710cd8640 1 -- 172.21.15.72:0/1167565600 learned_addr learned my addr 172.21.15.72:0/1167565600 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f070bfff640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c155890 0x7f070c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f07114d9640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f070c15f9b0 0x7f070c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f0710cd8640 1 -- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f070c15f9b0 msgr2=0x7f070c0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f0710cd8640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f070c15f9b0 0x7f070c0aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f0710cd8640 1 -- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c155890 msgr2=0x7f070c0af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f0710cd8640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c155890 0x7f070c0af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f0710cd8640 1 -- 172.21.15.72:0/1167565600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80094d0 con 0x7f070c1554b0 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f07114d9640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f070c15f9b0 0x7f070c0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f070bfff640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c155890 0x7f070c0af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.634+0000 7f0710cd8640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c1554b0 0x7f070c143fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f070000b420 tx=0x7f070000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:31.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.635+0000 7f07097fa640 1 -- 172.21.15.72:0/1167565600 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700011830 con 0x7f070c1554b0 2023-12-12T09:54:31.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.635+0000 7f07097fa640 1 -- 172.21.15.72:0/1167565600 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700011990 con 0x7f070c1554b0 2023-12-12T09:54:31.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.635+0000 7f07097fa640 1 -- 172.21.15.72:0/1167565600 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700011d40 con 0x7f070c1554b0 2023-12-12T09:54:31.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.635+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c141290 con 0x7f070c1554b0 2023-12-12T09:54:31.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.635+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c15cbf0 con 0x7f070c1554b0 2023-12-12T09:54:31.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.636+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c0744c0 con 0x7f070c1554b0 2023-12-12T09:54:31.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.637+0000 7f07097fa640 1 -- 172.21.15.72:0/1167565600 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f070001b460 con 0x7f070c1554b0 2023-12-12T09:54:31.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.638+0000 7f07097fa640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f06f006ae40 0x7f06f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.638+0000 7f07097fa640 1 -- 172.21.15.72:0/1167565600 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f0700089110 con 0x7f070c1554b0 2023-12-12T09:54:31.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.638+0000 7f070bfff640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f06f006ae40 0x7f06f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.639+0000 7f070bfff640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f06f006ae40 0x7f06f006d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06f8036000 tx=0x7f06f803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:31.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.643+0000 7f07097fa640 1 -- 172.21.15.72:0/1167565600 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0700054250 con 0x7f070c1554b0 2023-12-12T09:54:31.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:31.840+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f070c15c2f0 con 0x7f070c1554b0 2023-12-12T09:54:32.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.406+0000 7f07097fa640 1 -- 172.21.15.72:0/1167565600 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0700057870 con 0x7f070c1554b0 2023-12-12T09:54:32.409 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' created 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.409+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f06f006ae40 msgr2=0x7f06f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.409+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f06f006ae40 0x7f06f006d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06f8036000 tx=0x7f06f803d040 comp rx=0 tx=0).stop 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.409+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c1554b0 msgr2=0x7f070c143fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.409+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c1554b0 0x7f070c143fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f070000b420 tx=0x7f070000b8e0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.410+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 shutdown_connections 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.410+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f06f006ae40 0x7f06f006d300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.410+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f070c15f9b0 0x7f070c0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.410+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c155890 0x7f070c0af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.410+0000 7f0712f3c640 1 --2- 172.21.15.72:0/1167565600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f070c1554b0 0x7f070c143fe0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.410+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 >> 172.21.15.72:0/1167565600 conn(0x7f070c06a880 msgr2=0x7f070c06ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:32.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.410+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 shutdown_connections 2023-12-12T09:54:32.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.410+0000 7f0712f3c640 1 -- 172.21.15.72:0/1167565600 wait complete. 2023-12-12T09:54:32.452 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-12T09:54:32.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.634+0000 7feea9d78640 1 Processor -- start 2023-12-12T09:54:32.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.635+0000 7feea9d78640 1 -- start start 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7feea9d78640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea415af50 0x7feea406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7feea9d78640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7feea40ce7e0 0x7feea415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7feea9d78640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feea406a460 0x7feea406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7feea9d78640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea406ae30 con 0x7feea40ce7e0 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7feea9d78640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea40717e0 con 0x7feea406a460 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7feea9d78640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea4071cd0 con 0x7feea415af50 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7fee9affd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7feea40ce7e0 0x7feea415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7fee9affd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7feea40ce7e0 0x7feea415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54290/0 (socket says 172.21.15.72:54290) 2023-12-12T09:54:32.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.636+0000 7fee9affd640 1 -- 172.21.15.72:0/3685129902 learned_addr learned my addr 172.21.15.72:0/3685129902 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:32.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7feea3fff640 1 --2- 172.21.15.72:0/3685129902 >> v2:172.21.15.113:3300/0 conn(0x7feea406a460 0x7feea406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:32.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7fee9affd640 1 -- 172.21.15.72:0/3685129902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea415af50 msgr2=0x7feea406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:32.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7feea37fe640 1 --2- 172.21.15.72:0/3685129902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea415af50 0x7feea406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:32.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7fee9affd640 1 --2- 172.21.15.72:0/3685129902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea415af50 0x7feea406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7fee9affd640 1 -- 172.21.15.72:0/3685129902 >> v2:172.21.15.113:3300/0 conn(0x7feea406a460 msgr2=0x7feea406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:32.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7fee9affd640 1 --2- 172.21.15.72:0/3685129902 >> v2:172.21.15.113:3300/0 conn(0x7feea406a460 0x7feea406a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7fee9affd640 1 -- 172.21.15.72:0/3685129902 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea406c6b0 con 0x7feea40ce7e0 2023-12-12T09:54:32.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7feea3fff640 1 --2- 172.21.15.72:0/3685129902 >> v2:172.21.15.113:3300/0 conn(0x7feea406a460 0x7feea406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:32.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.637+0000 7feea37fe640 1 --2- 172.21.15.72:0/3685129902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea415af50 0x7feea406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:32.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.638+0000 7fee9affd640 1 --2- 172.21.15.72:0/3685129902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea40ce7e0 0x7feea415ec00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feea40cec40 tx=0x7fee8c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06a45bfe06c2ba6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:32.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.638+0000 7feea2ffd640 1 -- 172.21.15.72:0/3685129902 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c031750 con 0x7feea40ce7e0 2023-12-12T09:54:32.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.638+0000 7feea2ffd640 1 -- 172.21.15.72:0/3685129902 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c0318b0 con 0x7feea40ce7e0 2023-12-12T09:54:32.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.638+0000 7feea2ffd640 1 -- 172.21.15.72:0/3685129902 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c031c40 con 0x7feea40ce7e0 2023-12-12T09:54:32.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 -- 172.21.15.72:0/3685129902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea40ce7e0 msgr2=0x7feea415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:32.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 --2- 172.21.15.72:0/3685129902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea40ce7e0 0x7feea415ec00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feea40cec40 tx=0x7fee8c02b7f0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 -- 172.21.15.72:0/3685129902 shutdown_connections 2023-12-12T09:54:32.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 --2- 172.21.15.72:0/3685129902 >> v2:172.21.15.113:3300/0 conn(0x7feea406a460 0x7feea406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 --2- 172.21.15.72:0/3685129902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea40ce7e0 0x7feea415ec00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 --2- 172.21.15.72:0/3685129902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea415af50 0x7feea406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 -- 172.21.15.72:0/3685129902 >> 172.21.15.72:0/3685129902 conn(0x7feea40cee70 msgr2=0x7feea40cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:32.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 -- 172.21.15.72:0/3685129902 shutdown_connections 2023-12-12T09:54:32.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.639+0000 7feea9d78640 1 -- 172.21.15.72:0/3685129902 wait complete. 2023-12-12T09:54:32.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.641+0000 7feea9d78640 1 Processor -- start 2023-12-12T09:54:32.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.641+0000 7feea9d78640 1 -- start start 2023-12-12T09:54:32.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea9d78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea406a460 0x7feea41521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea9d78640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea40ce7e0 0x7feea409a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea9d78640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea415af50 0x7feea40b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea9d78640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea4152730 con 0x7feea415af50 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea9d78640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea409a630 con 0x7feea406a460 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea9d78640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea40c6d30 con 0x7feea40ce7e0 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea415af50 0x7feea40b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea415af50 0x7feea40b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54302/0 (socket says 172.21.15.72:54302) 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.642+0000 7feea3fff640 1 -- 172.21.15.72:0/3954754498 learned_addr learned my addr 172.21.15.72:0/3954754498 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea37fe640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea406a460 0x7feea41521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7fee9affd640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea40ce7e0 0x7feea409a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea3fff640 1 -- 172.21.15.72:0/3954754498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea40ce7e0 msgr2=0x7feea409a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea3fff640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea40ce7e0 0x7feea409a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea3fff640 1 -- 172.21.15.72:0/3954754498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea406a460 msgr2=0x7feea41521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:32.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea3fff640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea406a460 0x7feea41521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:32.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea3fff640 1 -- 172.21.15.72:0/3954754498 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c0094d0 con 0x7feea415af50 2023-12-12T09:54:32.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea3fff640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea415af50 0x7feea40b3db0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c7c0 tx=0x7fee9c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:32.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea0ff9640 1 -- 172.21.15.72:0/3954754498 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c004230 con 0x7feea415af50 2023-12-12T09:54:32.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea0ff9640 1 -- 172.21.15.72:0/3954754498 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c004390 con 0x7feea415af50 2023-12-12T09:54:32.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea40b19d0 con 0x7feea415af50 2023-12-12T09:54:32.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.643+0000 7feea0ff9640 1 -- 172.21.15.72:0/3954754498 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c007860 con 0x7feea415af50 2023-12-12T09:54:32.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.644+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea40b1f90 con 0x7feea415af50 2023-12-12T09:54:32.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.645+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea4074450 con 0x7feea415af50 2023-12-12T09:54:32.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.646+0000 7feea0ff9640 1 -- 172.21.15.72:0/3954754498 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee9c0079c0 con 0x7feea415af50 2023-12-12T09:54:32.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.647+0000 7feea0ff9640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee8406ae40 0x7fee8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:32.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.647+0000 7feea0ff9640 1 -- 172.21.15.72:0/3954754498 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fee9c089a20 con 0x7feea415af50 2023-12-12T09:54:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.647+0000 7feea37fe640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee8406ae40 0x7fee8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:32.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.648+0000 7feea37fe640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee8406ae40 0x7fee8406d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee94007fb0 tx=0x7fee94007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:32.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.652+0000 7feea0ff9640 1 -- 172.21.15.72:0/3954754498 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee9c054ac0 con 0x7feea415af50 2023-12-12T09:54:32.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:32.880+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7feea40c2390 con 0x7feea415af50 2023-12-12T09:54:33.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.410+0000 7feea0ff9640 1 -- 172.21.15.72:0/3954754498 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fee9c0150c0 con 0x7feea415af50 2023-12-12T09:54:33.415 INFO:teuthology.orchestra.run.smithi072.stderr:new fs with metadata pool 7 and data pool 8 2023-12-12T09:54:33.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.414+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee8406ae40 msgr2=0x7fee8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.414+0000 7feea9d78640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee8406ae40 0x7fee8406d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee94007fb0 tx=0x7fee94007990 comp rx=0 tx=0).stop 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.414+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea415af50 msgr2=0x7feea40b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.414+0000 7feea9d78640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea415af50 0x7feea40b3db0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c7c0 tx=0x7fee9c00cc80 comp rx=0 tx=0).stop 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.415+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 shutdown_connections 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.415+0000 7feea9d78640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fee8406ae40 0x7fee8406d300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.415+0000 7feea9d78640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feea415af50 0x7feea40b3db0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.415+0000 7feea9d78640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feea40ce7e0 0x7feea409a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.415+0000 7feea9d78640 1 --2- 172.21.15.72:0/3954754498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feea406a460 0x7feea41521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.415+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 >> 172.21.15.72:0/3954754498 conn(0x7feea40cee70 msgr2=0x7feea40cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:33.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.416+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 shutdown_connections 2023-12-12T09:54:33.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.416+0000 7feea9d78640 1 -- 172.21.15.72:0/3954754498 wait complete. 2023-12-12T09:54:33.460 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:33.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.640+0000 7f11c99e5640 1 Processor -- start 2023-12-12T09:54:33.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.640+0000 7f11c99e5640 1 -- start start 2023-12-12T09:54:33.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c99e5640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f11c4143e00 0x7f11c40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:33.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c99e5640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c40b3bb0 con 0x7f11c406c410 2023-12-12T09:54:33.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c99e5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c41441e0 con 0x7f11c4072180 2023-12-12T09:54:33.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c99e5640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c4072680 con 0x7f11c4143e00 2023-12-12T09:54:33.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c2ffd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f11c4143e00 0x7f11c40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:33.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c37fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f11c406c410 0x7f11c415ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35758/0 (socket says 172.21.15.72:35758) 2023-12-12T09:54:33.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c37fe640 1 -- 172.21.15.72:0/3951490855 learned_addr learned my addr 172.21.15.72:0/3951490855 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:33.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c2ffd640 1 -- 172.21.15.72:0/3951490855 >> v1:172.21.15.113:6789/0 conn(0x7f11c4072180 legacy=0x7f11c40b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:33.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c2ffd640 1 -- 172.21.15.72:0/3951490855 >> v1:172.21.15.72:6789/0 conn(0x7f11c406c410 legacy=0x7f11c415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:33.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.643+0000 7f11c2ffd640 1 -- 172.21.15.72:0/3951490855 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40b1e30 con 0x7f11c4143e00 2023-12-12T09:54:33.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.642+0000 7f11c27fc640 1 -- 172.21.15.72:0/3951490855 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970869702 0 0) 0x7f11c40b3bb0 con 0x7f11c406c410 2023-12-12T09:54:33.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.644+0000 7f11c2ffd640 1 --2- 172.21.15.72:0/3951490855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c4143e00 0x7f11c40b4040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f11b40097a0 tx=0x7f11b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8458c5cbb1cbd9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:33.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.644+0000 7f11c27fc640 1 -- 172.21.15.72:0/3951490855 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4031830 con 0x7f11c4143e00 2023-12-12T09:54:33.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.644+0000 7f11c27fc640 1 -- 172.21.15.72:0/3951490855 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b4031990 con 0x7f11c4143e00 2023-12-12T09:54:33.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.644+0000 7f11c27fc640 1 -- 172.21.15.72:0/3951490855 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4031d20 con 0x7f11c4143e00 2023-12-12T09:54:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.645+0000 7f11c99e5640 1 -- 172.21.15.72:0/3951490855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c4143e00 msgr2=0x7f11c40b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.645+0000 7f11c99e5640 1 --2- 172.21.15.72:0/3951490855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c4143e00 0x7f11c40b4040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f11b40097a0 tx=0x7f11b402b8d0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.645+0000 7f11c99e5640 1 -- 172.21.15.72:0/3951490855 shutdown_connections 2023-12-12T09:54:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.645+0000 7f11c99e5640 1 --2- 172.21.15.72:0/3951490855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c4143e00 0x7f11c40b4040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.645+0000 7f11c99e5640 1 -- 172.21.15.72:0/3951490855 >> 172.21.15.72:0/3951490855 conn(0x7f11c40b0a90 msgr2=0x7f11c40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.645+0000 7f11c99e5640 1 -- 172.21.15.72:0/3951490855 shutdown_connections 2023-12-12T09:54:33.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.645+0000 7f11c99e5640 1 -- 172.21.15.72:0/3951490855 wait complete. 2023-12-12T09:54:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.647+0000 7f11c99e5640 1 Processor -- start 2023-12-12T09:54:33.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.647+0000 7f11c99e5640 1 -- start start 2023-12-12T09:54:33.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c99e5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c406c410 0x7f11c4137b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c99e5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c4072180 0x7f11c41501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c99e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11c4143e00 0x7f11c40af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c99e5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4151870 con 0x7f11c406c410 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c99e5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4150c60 con 0x7f11c4072180 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c99e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4150f40 con 0x7f11c4143e00 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c406c410 0x7f11c4137b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c406c410 0x7f11c4137b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54316/0 (socket says 172.21.15.72:54316) 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.648+0000 7f11c2ffd640 1 -- 172.21.15.72:0/3754724577 learned_addr learned my addr 172.21.15.72:0/3754724577 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c2ffd640 1 -- 172.21.15.72:0/3754724577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11c4143e00 msgr2=0x7f11c40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c2ffd640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11c4143e00 0x7f11c40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c2ffd640 1 -- 172.21.15.72:0/3754724577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c4072180 msgr2=0x7f11c41501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11ba7fc640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c4072180 0x7f11c41501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c2ffd640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c4072180 0x7f11c41501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c2ffd640 1 -- 172.21.15.72:0/3754724577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4009400 con 0x7f11c406c410 2023-12-12T09:54:33.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c2ffd640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c406c410 0x7f11c4137b20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f11b4000c00 tx=0x7f11b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:33.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c0ff9640 1 -- 172.21.15.72:0/3754724577 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b403aa60 con 0x7f11c406c410 2023-12-12T09:54:33.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c0ff9640 1 -- 172.21.15.72:0/3754724577 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b403abc0 con 0x7f11c406c410 2023-12-12T09:54:33.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c0ff9640 1 -- 172.21.15.72:0/3754724577 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b40443f0 con 0x7f11c406c410 2023-12-12T09:54:33.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.649+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c4151220 con 0x7f11c406c410 2023-12-12T09:54:33.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.650+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c415b020 con 0x7f11c406c410 2023-12-12T09:54:33.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.651+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c40745d0 con 0x7f11c406c410 2023-12-12T09:54:33.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.651+0000 7f11c0ff9640 1 -- 172.21.15.72:0/3754724577 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11b404d450 con 0x7f11c406c410 2023-12-12T09:54:33.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.652+0000 7f11c0ff9640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11a406ae40 0x7f11a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:33.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.652+0000 7f11c0ff9640 1 -- 172.21.15.72:0/3754724577 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f11b40ab9e0 con 0x7f11c406c410 2023-12-12T09:54:33.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.653+0000 7f11ba7fc640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11a406ae40 0x7f11a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:33.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.653+0000 7f11ba7fc640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11a406ae40 0x7f11a406d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11c41533e0 tx=0x7f11ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:33.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.658+0000 7f11c0ff9640 1 -- 172.21.15.72:0/3754724577 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11b40768b0 con 0x7f11c406c410 2023-12-12T09:54:33.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.836+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11c406ddc0 con 0x7f11c406c410 2023-12-12T09:54:33.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.839+0000 7f11c0ff9640 1 -- 172.21.15.72:0/3754724577 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f11b4079ed0 con 0x7f11c406c410 2023-12-12T09:54:33.841 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:33.841 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":50,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:32.881954+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:54:30.694215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-12T09:54:31.894291+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:33.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11a406ae40 msgr2=0x7f11a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:33.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11a406ae40 0x7f11a406d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11c41533e0 tx=0x7f11ac009040 comp rx=0 tx=0).stop 2023-12-12T09:54:33.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c406c410 msgr2=0x7f11c4137b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:33.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c406c410 0x7f11c4137b20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f11b4000c00 tx=0x7f11b40046c0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 shutdown_connections 2023-12-12T09:54:33.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f11a406ae40 0x7f11a406d300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11c4143e00 0x7f11c40af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c4072180 0x7f11c41501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 --2- 172.21.15.72:0/3754724577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11c406c410 0x7f11c4137b20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:33.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 >> 172.21.15.72:0/3754724577 conn(0x7f11c40b0a90 msgr2=0x7f11c4144400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:33.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.841+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 shutdown_connections 2023-12-12T09:54:33.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:33.842+0000 7f11c99e5640 1 -- 172.21.15.72:0/3754724577 wait complete. 2023-12-12T09:54:33.861 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:34.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.067+0000 7fc00f361640 1 Processor -- start 2023-12-12T09:54:34.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.067+0000 7fc00f361640 1 -- start start 2023-12-12T09:54:34.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.069+0000 7fc00f361640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc008157a60 0x7fc00806c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.069+0000 7fc00f361640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00806ca40 con 0x7fc00815e2b0 2023-12-12T09:54:34.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.069+0000 7fc00f361640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc008071cd0 con 0x7fc00806de20 2023-12-12T09:54:34.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.069+0000 7fc00f361640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc008070f10 con 0x7fc008157a60 2023-12-12T09:54:34.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.069+0000 7fc00d8fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc00815e2b0 0x7fc0080cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35766/0 (socket says 172.21.15.72:35766) 2023-12-12T09:54:34.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.069+0000 7fc00d8fe640 1 -- 172.21.15.72:0/1769534455 learned_addr learned my addr 172.21.15.72:0/1769534455 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:34.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.069+0000 7fc00d0fd640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc008157a60 0x7fc00806c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106571167 0 0) 0x7fc00806ca40 con 0x7fc00815e2b0 2023-12-12T09:54:34.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff4005730 con 0x7fc00815e2b0 2023-12-12T09:54:34.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3432139389 0 0) 0x7fbff4005730 con 0x7fc00815e2b0 2023-12-12T09:54:34.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc008157a60 msgr2=0x7fc00806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:34.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc008157a60 0x7fc00806c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 >> v1:172.21.15.113:6789/0 conn(0x7fc00806de20 legacy=0x7fc00815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0080711f0 con 0x7fc00815e2b0 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fc0000019f0 con 0x7fc00815e2b0 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 >> v1:172.21.15.72:6789/0 conn(0x7fc00815e2b0 legacy=0x7fc0080cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fbff4005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc008157a60 0x7fbff40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbff4006f10 0x7fbff4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc008070f10 con 0x7fbff4005bb0 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff400dcc0 con 0x7fbff4006f10 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff400de80 con 0x7fc008157a60 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.070+0000 7fc00d0fd640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fbff4005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.071+0000 7fc00c8fc640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc008157a60 0x7fbff40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.071+0000 7fc00d0fd640 1 -- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc008157a60 msgr2=0x7fbff40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.071+0000 7fc00d0fd640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc008157a60 0x7fbff40069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.071+0000 7fc00d8fe640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbff4006f10 0x7fbff4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.071+0000 7fc00d0fd640 1 -- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbff4006f10 msgr2=0x7fbff4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:34.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.071+0000 7fc00d0fd640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbff4006f10 0x7fbff4007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.071+0000 7fc00d0fd640 1 -- 172.21.15.72:0/1769534455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4005730 con 0x7fbff4005bb0 2023-12-12T09:54:34.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.071+0000 7fc00d8fe640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbff4006f10 0x7fbff4007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.072+0000 7fc00d0fd640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fbff4005fd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc004009ac0 tx=0x7fbff4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3d0742b85d2c1a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.072+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00402c600 con 0x7fbff4005bb0 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.072+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00402c760 con 0x7fbff4005bb0 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.072+0000 7fbffffff640 1 -- 172.21.15.72:0/1769534455 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00402caf0 con 0x7fbff4005bb0 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.073+0000 7fc00f361640 1 -- 172.21.15.72:0/1769534455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 msgr2=0x7fbff4005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.073+0000 7fc00f361640 1 --2- 172.21.15.72:0/1769534455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fbff4005fd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc004009ac0 tx=0x7fbff4007a00 comp rx=0 tx=0).stop 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.073+0000 7fc00d8fe640 1 -- 172.21.15.72:0/1769534455 reap_dead start 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.073+0000 7fc00f361640 1 -- 172.21.15.72:0/1769534455 shutdown_connections 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.073+0000 7fc00f361640 1 -- 172.21.15.72:0/1769534455 >> 172.21.15.72:0/1769534455 conn(0x7fc0080b0a90 msgr2=0x7fc0080b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.073+0000 7fc00f361640 1 -- 172.21.15.72:0/1769534455 shutdown_connections 2023-12-12T09:54:34.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.073+0000 7fc00f361640 1 -- 172.21.15.72:0/1769534455 wait complete. 2023-12-12T09:54:34.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.075+0000 7fc00f361640 1 Processor -- start 2023-12-12T09:54:34.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.075+0000 7fc00f361640 1 -- start start 2023-12-12T09:54:34.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00f361640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fc0080b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00f361640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0081f0030 0x7fc0081539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00f361640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0080cdb50 0x7fc00814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00f361640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080b4e20 con 0x7fbff4005bb0 2023-12-12T09:54:34.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00f361640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0081f04b0 con 0x7fc0081f0030 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00f361640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc008153f30 con 0x7fc0080cdb50 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00d0fd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fc0080b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00d0fd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fc0080b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54328/0 (socket says 172.21.15.72:54328) 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00d0fd640 1 -- 172.21.15.72:0/1538135756 learned_addr learned my addr 172.21.15.72:0/1538135756 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00d0fd640 1 -- 172.21.15.72:0/1538135756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0080cdb50 msgr2=0x7fc00814ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00d0fd640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0080cdb50 0x7fc00814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00d0fd640 1 -- 172.21.15.72:0/1538135756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0081f0030 msgr2=0x7fc0081539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00d0fd640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0081f0030 0x7fc0081539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.076+0000 7fc00d0fd640 1 -- 172.21.15.72:0/1538135756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0080b4580 con 0x7fbff4005bb0 2023-12-12T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.077+0000 7fc00d0fd640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fc0080b29f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc00402c1a0 tx=0x7fc004004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:34.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.077+0000 7fbffdffb640 1 -- 172.21.15.72:0/1538135756 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004003810 con 0x7fbff4005bb0 2023-12-12T09:54:34.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.077+0000 7fbffdffb640 1 -- 172.21.15.72:0/1538135756 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004003970 con 0x7fbff4005bb0 2023-12-12T09:54:34.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.077+0000 7fbffdffb640 1 -- 172.21.15.72:0/1538135756 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004003ce0 con 0x7fbff4005bb0 2023-12-12T09:54:34.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.077+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0080b4860 con 0x7fbff4005bb0 2023-12-12T09:54:34.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.077+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0080b4270 con 0x7fbff4005bb0 2023-12-12T09:54:34.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.079+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0080745d0 con 0x7fbff4005bb0 2023-12-12T09:54:34.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.079+0000 7fbffdffb640 1 -- 172.21.15.72:0/1538135756 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc004034440 con 0x7fbff4005bb0 2023-12-12T09:54:34.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.080+0000 7fbffdffb640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbfec06ae40 0x7fbfec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.080+0000 7fbffdffb640 1 -- 172.21.15.72:0/1538135756 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc0040ab580 con 0x7fbff4005bb0 2023-12-12T09:54:34.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.080+0000 7fc00c8fc640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbfec06ae40 0x7fbfec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.081+0000 7fc00c8fc640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbfec06ae40 0x7fbfec06d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbff8007fd0 tx=0x7fbff8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:34.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.085+0000 7fbffdffb640 1 -- 172.21.15.72:0/1538135756 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc004076610 con 0x7fbff4005bb0 2023-12-12T09:54:34.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.275+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc008151ec0 con 0x7fbff4005bb0 2023-12-12T09:54:34.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.278+0000 7fbffdffb640 1 -- 172.21.15.72:0/1538135756 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16787 (secure 0 0 0) 0x7fc0040044e0 con 0x7fbff4005bb0 2023-12-12T09:54:34.281 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:34.281 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":50,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:32.881954+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:54:30.694215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-12T09:54:31.894291+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:34.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbfec06ae40 msgr2=0x7fbfec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbfec06ae40 0x7fbfec06d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbff8007fd0 tx=0x7fbff8009040 comp rx=0 tx=0).stop 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 msgr2=0x7fc0080b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fc0080b29f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc00402c1a0 tx=0x7fc004004730 comp rx=0 tx=0).stop 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 shutdown_connections 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbfec06ae40 0x7fbfec06d300 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0080cdb50 0x7fc00814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0081f0030 0x7fc0081539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 --2- 172.21.15.72:0/1538135756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbff4005bb0 0x7fc0080b29f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 >> 172.21.15.72:0/1538135756 conn(0x7fc0080b0a90 msgr2=0x7fc0080701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.281+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 shutdown_connections 2023-12-12T09:54:34.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.282+0000 7fc00f361640 1 -- 172.21.15.72:0/1538135756 wait complete. 2023-12-12T09:54:34.303 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-12T09:54:34.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.510+0000 7f942c3e5640 1 Processor -- start 2023-12-12T09:54:34.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.510+0000 7f942c3e5640 1 -- start start 2023-12-12T09:54:34.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f942c3e5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9424142de0 0x7f94240cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f942c3e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f942406dc10 0x7f9424070250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f942c3e5640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94240b4980 con 0x7f94241432b0 2023-12-12T09:54:34.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f942c3e5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94240ce2a0 con 0x7f9424142de0 2023-12-12T09:54:34.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f942c3e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94240b4050 con 0x7f942406dc10 2023-12-12T09:54:34.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f9429980640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f94241432b0 0x7f942406d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:35774/0 (socket says 172.21.15.72:35774) 2023-12-12T09:54:34.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f9429980640 1 -- 172.21.15.72:0/242495341 learned_addr learned my addr 172.21.15.72:0/242495341 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:34.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f942a181640 1 --2- 172.21.15.72:0/242495341 >> v2:172.21.15.113:3300/0 conn(0x7f9424142de0 0x7f94240cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46311810 0 0) 0x7f94240b4980 con 0x7f94241432b0 2023-12-12T09:54:34.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.512+0000 7f942a982640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f942406dc10 0x7f9424070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9410003520 con 0x7f94241432b0 2023-12-12T09:54:34.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3294583315 0 0) 0x7f9410003520 con 0x7f94241432b0 2023-12-12T09:54:34.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f942406dc10 msgr2=0x7f9424070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:34.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f942406dc10 0x7f9424070250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 >> v2:172.21.15.113:3300/0 conn(0x7f9424142de0 msgr2=0x7f94240cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 --2- 172.21.15.72:0/242495341 >> v2:172.21.15.113:3300/0 conn(0x7f9424142de0 0x7f94240cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240b4330 con 0x7f94241432b0 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f9414001670 con 0x7f94241432b0 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 >> v1:172.21.15.72:6789/0 conn(0x7f94241432b0 legacy=0x7f942406d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410003970 0x7f9410003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f942406dc10 0x7f94100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9410004d00 0x7f9410005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94240ce2a0 con 0x7f9410004d00 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94240b4050 con 0x7f9410003970 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941000b9d0 con 0x7f942406dc10 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.513+0000 7f942a982640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9410004d00 0x7f9410005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.514+0000 7f942a181640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410003970 0x7f9410003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.514+0000 7f9429980640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f942406dc10 0x7f94100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.514+0000 7f942a982640 1 -- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f942406dc10 msgr2=0x7f94100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:34.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.514+0000 7f942a982640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f942406dc10 0x7f94100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.514+0000 7f942a982640 1 -- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410003970 msgr2=0x7f9410003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:34.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.514+0000 7f942a982640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410003970 0x7f9410003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.514+0000 7f942a982640 1 -- 172.21.15.72:0/242495341 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410003520 con 0x7f9410004d00 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.515+0000 7f942a982640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9410004d00 0x7f9410005180 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e170 tx=0x7f94100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b856d102b60fbe25 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.515+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c02cbf0 con 0x7f9410004d00 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.515+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c00f040 con 0x7f9410004d00 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.515+0000 7f942917f640 1 -- 172.21.15.72:0/242495341 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c034440 con 0x7f9410004d00 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.516+0000 7f942c3e5640 1 -- 172.21.15.72:0/242495341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9410004d00 msgr2=0x7f9410005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.516+0000 7f942c3e5640 1 --2- 172.21.15.72:0/242495341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9410004d00 0x7f9410005180 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e170 tx=0x7f94100057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.516+0000 7f942a982640 1 -- 172.21.15.72:0/242495341 reap_dead start 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.516+0000 7f942c3e5640 1 -- 172.21.15.72:0/242495341 shutdown_connections 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.516+0000 7f942c3e5640 1 -- 172.21.15.72:0/242495341 >> 172.21.15.72:0/242495341 conn(0x7f94240cb850 msgr2=0x7f94240cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.516+0000 7f942c3e5640 1 -- 172.21.15.72:0/242495341 shutdown_connections 2023-12-12T09:54:34.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.516+0000 7f942c3e5640 1 -- 172.21.15.72:0/242495341 wait complete. 2023-12-12T09:54:34.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.518+0000 7f942c3e5640 1 Processor -- start 2023-12-12T09:54:34.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.518+0000 7f942c3e5640 1 -- start start 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f942c3e5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410004d00 0x7f942412dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f942c3e5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9424071fa0 0x7f94240a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f942c3e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94240a8990 0x7f942412b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f942c3e5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94240a9510 con 0x7f9424071fa0 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f942c3e5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94240a8e10 con 0x7f9410004d00 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f942c3e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942412e110 con 0x7f94240a8990 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f9429980640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9424071fa0 0x7f94240a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f9429980640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9424071fa0 0x7f94240a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54352/0 (socket says 172.21.15.72:54352) 2023-12-12T09:54:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.519+0000 7f9429980640 1 -- 172.21.15.72:0/3674443803 learned_addr learned my addr 172.21.15.72:0/3674443803 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f942a181640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410004d00 0x7f942412dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f9429980640 1 -- 172.21.15.72:0/3674443803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94240a8990 msgr2=0x7f942412b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f9429980640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94240a8990 0x7f942412b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f9429980640 1 -- 172.21.15.72:0/3674443803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410004d00 msgr2=0x7f942412dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f9429980640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410004d00 0x7f942412dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f9429980640 1 -- 172.21.15.72:0/3674443803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424070f00 con 0x7f9424071fa0 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f9429980640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9424071fa0 0x7f94240a8450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f941400e8c0 tx=0x7f941400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f941affd640 1 -- 172.21.15.72:0/3674443803 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414017070 con 0x7f9424071fa0 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f941affd640 1 -- 172.21.15.72:0/3674443803 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941400c850 con 0x7f9424071fa0 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94240711e0 con 0x7f9424071fa0 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f941affd640 1 -- 172.21.15.72:0/3674443803 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941400ca00 con 0x7f9424071fa0 2023-12-12T09:54:34.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.520+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94240717a0 con 0x7f9424071fa0 2023-12-12T09:54:34.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.521+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94240745d0 con 0x7f9424071fa0 2023-12-12T09:54:34.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.523+0000 7f941affd640 1 -- 172.21.15.72:0/3674443803 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f941400cb60 con 0x7f9424071fa0 2023-12-12T09:54:34.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.523+0000 7f941affd640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f940806ae40 0x7f940806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:34.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.524+0000 7f941affd640 1 -- 172.21.15.72:0/3674443803 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9414088fe0 con 0x7f9424071fa0 2023-12-12T09:54:34.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.524+0000 7f942a181640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f940806ae40 0x7f940806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:34.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.524+0000 7f942a181640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f940806ae40 0x7f940806d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9420009950 tx=0x7f9420008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:34.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.528+0000 7f941affd640 1 -- 172.21.15.72:0/3674443803 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9414053eb0 con 0x7f9424071fa0 2023-12-12T09:54:34.585 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:54:34.594 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:54:34.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:34.760+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f94240cc650 con 0x7f9424071fa0 2023-12-12T09:54:35.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.461+0000 7f941affd640 1 -- 172.21.15.72:0/3674443803 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f94140574d0 con 0x7f9424071fa0 2023-12-12T09:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f940806ae40 msgr2=0x7f940806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f940806ae40 0x7f940806d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9420009950 tx=0x7f9420008040 comp rx=0 tx=0).stop 2023-12-12T09:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9424071fa0 msgr2=0x7f94240a8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9424071fa0 0x7f94240a8450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f941400e8c0 tx=0x7f941400ed80 comp rx=0 tx=0).stop 2023-12-12T09:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 shutdown_connections 2023-12-12T09:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f940806ae40 0x7f940806d300 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94240a8990 0x7f942412b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9424071fa0 0x7f94240a8450 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 --2- 172.21.15.72:0/3674443803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9410004d00 0x7f942412dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 >> 172.21.15.72:0/3674443803 conn(0x7f94240cb850 msgr2=0x7f94240aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 shutdown_connections 2023-12-12T09:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.465+0000 7f942c3e5640 1 -- 172.21.15.72:0/3674443803 wait complete. 2023-12-12T09:54:35.487 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-12T09:54:35.487 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-12T09:54:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.694+0000 7f1d0044f640 1 Processor -- start 2023-12-12T09:54:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.694+0000 7f1d0044f640 1 -- start start 2023-12-12T09:54:35.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1d0044f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cf806d770 0x7f1cf80b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:35.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1d0044f640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1cf8141960 0x7f1cf80cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:35.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1d0044f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf815b780 con 0x7f1cf806db50 2023-12-12T09:54:35.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1d0044f640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf815b9d0 con 0x7f1cf8141960 2023-12-12T09:54:35.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1d0044f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf815bcb0 con 0x7f1cf806d770 2023-12-12T09:54:35.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1cfe9ec640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1cf8141960 0x7f1cf80cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:35.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1cfe9ec640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1cf8141960 0x7f1cf80cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49808/0 (socket says 172.21.15.72:49808) 2023-12-12T09:54:35.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1cfe9ec640 1 -- 172.21.15.72:0/238158973 learned_addr learned my addr 172.21.15.72:0/238158973 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:35.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1cfe1eb640 1 --2- 172.21.15.72:0/238158973 >> v2:172.21.15.113:3300/0 conn(0x7f1cf806d770 0x7f1cf80b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:35.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.696+0000 7f1cfe9ec640 1 -- 172.21.15.72:0/238158973 >> v2:172.21.15.113:3300/0 conn(0x7f1cf806d770 msgr2=0x7f1cf80b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:35.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.697+0000 7f1cfe9ec640 1 --2- 172.21.15.72:0/238158973 >> v2:172.21.15.113:3300/0 conn(0x7f1cf806d770 0x7f1cf80b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.697+0000 7f1cfe9ec640 1 -- 172.21.15.72:0/238158973 >> v1:172.21.15.113:6789/0 conn(0x7f1cf806db50 legacy=0x7f1cf8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:35.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.697+0000 7f1cfe9ec640 1 -- 172.21.15.72:0/238158973 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf815bf90 con 0x7f1cf8141960 2023-12-12T09:54:35.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.697+0000 7f1cfe1eb640 1 --2- 172.21.15.72:0/238158973 >> v2:172.21.15.113:3300/0 conn(0x7f1cf806d770 0x7f1cf80b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:35.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.698+0000 7f1cfe9ec640 1 --2- 172.21.15.72:0/238158973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf80cce00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80b1aa0 tx=0x7f1cf002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436ca24b9474d815 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:35.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.698+0000 7f1cfd1e9640 1 -- 172.21.15.72:0/238158973 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf002ca80 con 0x7f1cf8141960 2023-12-12T09:54:35.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.698+0000 7f1cfd1e9640 1 -- 172.21.15.72:0/238158973 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf002cbe0 con 0x7f1cf8141960 2023-12-12T09:54:35.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.698+0000 7f1cfd1e9640 1 -- 172.21.15.72:0/238158973 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00343f0 con 0x7f1cf8141960 2023-12-12T09:54:35.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.699+0000 7f1d0044f640 1 -- 172.21.15.72:0/238158973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 msgr2=0x7f1cf80cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:35.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.699+0000 7f1d0044f640 1 --2- 172.21.15.72:0/238158973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf80cce00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80b1aa0 tx=0x7f1cf002bee0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.699+0000 7f1d0044f640 1 -- 172.21.15.72:0/238158973 shutdown_connections 2023-12-12T09:54:35.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.699+0000 7f1d0044f640 1 --2- 172.21.15.72:0/238158973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf80cce00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.699+0000 7f1d0044f640 1 --2- 172.21.15.72:0/238158973 >> v2:172.21.15.113:3300/0 conn(0x7f1cf806d770 0x7f1cf80b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.699+0000 7f1d0044f640 1 -- 172.21.15.72:0/238158973 >> 172.21.15.72:0/238158973 conn(0x7f1cf80cb850 msgr2=0x7f1cf80cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:35.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.699+0000 7f1d0044f640 1 -- 172.21.15.72:0/238158973 shutdown_connections 2023-12-12T09:54:35.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.699+0000 7f1d0044f640 1 -- 172.21.15.72:0/238158973 wait complete. 2023-12-12T09:54:35.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.701+0000 7f1d0044f640 1 Processor -- start 2023-12-12T09:54:35.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.701+0000 7f1d0044f640 1 -- start start 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1d0044f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cf806d770 0x7f1cf814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1d0044f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf806db50 0x7f1cf813e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1d0044f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf814dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1d0044f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf815a950 con 0x7f1cf8141960 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1d0044f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf813eb50 con 0x7f1cf806db50 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1d0044f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf814cb60 con 0x7f1cf806d770 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1cfe9ec640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf814dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1cfe9ec640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf814dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49824/0 (socket says 172.21.15.72:49824) 2023-12-12T09:54:35.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1cfe9ec640 1 -- 172.21.15.72:0/34558542 learned_addr learned my addr 172.21.15.72:0/34558542 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:35.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1cfe9ec640 1 -- 172.21.15.72:0/34558542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cf806d770 msgr2=0x7f1cf814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:35.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1cfe9ec640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cf806d770 0x7f1cf814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1cfe9ec640 1 -- 172.21.15.72:0/34558542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf806db50 msgr2=0x7f1cf813e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.702+0000 7f1cfe9ec640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf806db50 0x7f1cf813e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.703+0000 7f1cfe9ec640 1 -- 172.21.15.72:0/34558542 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0009b30 con 0x7f1cf8141960 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.703+0000 7f1cfe9ec640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf814dc00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0036040 tx=0x7f1cf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.703+0000 7f1ceeffd640 1 -- 172.21.15.72:0/34558542 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0040670 con 0x7f1cf8141960 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.703+0000 7f1ceeffd640 1 -- 172.21.15.72:0/34558542 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf00407f0 con 0x7f1cf8141960 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.703+0000 7f1ceeffd640 1 -- 172.21.15.72:0/34558542 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0040ba0 con 0x7f1cf8141960 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.703+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf80b3940 con 0x7f1cf8141960 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.703+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf814d120 con 0x7f1cf8141960 2023-12-12T09:54:35.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.704+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf80745d0 con 0x7f1cf8141960 2023-12-12T09:54:35.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.705+0000 7f1ceeffd640 1 -- 172.21.15.72:0/34558542 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cf002cd50 con 0x7f1cf8141960 2023-12-12T09:54:35.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.706+0000 7f1ceeffd640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1cdc06ae40 0x7f1cdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:35.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.706+0000 7f1ceeffd640 1 -- 172.21.15.72:0/34558542 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1cf00ac6e0 con 0x7f1cf8141960 2023-12-12T09:54:35.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.706+0000 7f1cfe1eb640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1cdc06ae40 0x7f1cdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:35.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.707+0000 7f1cfe1eb640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1cdc06ae40 0x7f1cdc06d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006fd0 tx=0x7f1cf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:35.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.711+0000 7f1ceeffd640 1 -- 172.21.15.72:0/34558542 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cf00775b0 con 0x7f1cf8141960 2023-12-12T09:54:35.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:35.938+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1cf8072770 con 0x7f1cf8141960 2023-12-12T09:54:36.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.475+0000 7f1ceeffd640 1 -- 172.21.15.72:0/34558542 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1cf007abd0 con 0x7f1cf8141960 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.478+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1cdc06ae40 msgr2=0x7f1cdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1cdc06ae40 0x7f1cdc06d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006fd0 tx=0x7f1cf4008040 comp rx=0 tx=0).stop 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 msgr2=0x7f1cf814dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf814dc00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0036040 tx=0x7f1cf0004a40 comp rx=0 tx=0).stop 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 shutdown_connections 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f1cdc06ae40 0x7f1cdc06d300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf8141960 0x7f1cf814dc00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf806db50 0x7f1cf813e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 --2- 172.21.15.72:0/34558542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cf806d770 0x7f1cf814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 >> 172.21.15.72:0/34558542 conn(0x7f1cf80cb850 msgr2=0x7f1cf813f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:36.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 shutdown_connections 2023-12-12T09:54:36.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.479+0000 7f1d0044f640 1 -- 172.21.15.72:0/34558542 wait complete. 2023-12-12T09:54:36.500 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-12T09:54:36.500 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:36.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.704+0000 7f3ce6d1d640 1 Processor -- start 2023-12-12T09:54:36.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.704+0000 7f3ce6d1d640 1 -- start start 2023-12-12T09:54:36.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.705+0000 7f3ce6d1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ce006c410 0x7f3ce0166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:36.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.705+0000 7f3ce6d1d640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00b3bb0 con 0x7f3ce0143e00 2023-12-12T09:54:36.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.706+0000 7f3ce6d1d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00b4e60 con 0x7f3ce0072180 2023-12-12T09:54:36.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.706+0000 7f3ce6d1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce01441e0 con 0x7f3ce006c410 2023-12-12T09:54:36.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.706+0000 7f3ce4ab9640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f3ce0143e00 0x7f3ce00b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:60550/0 (socket says 172.21.15.72:60550) 2023-12-12T09:54:36.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.706+0000 7f3ce4ab9640 1 -- 172.21.15.72:0/4064479312 learned_addr learned my addr 172.21.15.72:0/4064479312 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.706+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ce006c410 0x7f3ce0166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.706+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048423073 0 0) 0x7f3ce00b3bb0 con 0x7f3ce0143e00 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.706+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd0003520 con 0x7f3ce0143e00 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100660572 0 0) 0x7f3cd0003520 con 0x7f3ce0143e00 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ce006c410 msgr2=0x7f3ce0166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ce006c410 0x7f3ce0166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 >> v1:172.21.15.113:6789/0 conn(0x7f3ce0072180 legacy=0x7f3ce00b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0072680 con 0x7f3ce0143e00 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f3cd4001670 con 0x7f3ce0143e00 2023-12-12T09:54:36.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 >> v1:172.21.15.72:6789/0 conn(0x7f3ce0143e00 legacy=0x7f3ce00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd00039a0 0x7f3cd0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd00047c0 0x7f3cd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cd0005180 0x7f3cd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce01441e0 con 0x7f3cd0005180 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd000c080 con 0x7f3cd00039a0 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd000c240 con 0x7f3cd00047c0 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ce006c410 0x7f3ce0166020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cd0005180 0x7f3cd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3ce4ab9640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd00039a0 0x7f3cd0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.707+0000 7f3cdffff640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd00047c0 0x7f3cd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.708+0000 7f3ce52ba640 1 -- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd00047c0 msgr2=0x7f3cd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.708+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd00047c0 0x7f3cd0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.708+0000 7f3ce52ba640 1 -- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd00039a0 msgr2=0x7f3cd0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.708+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd00039a0 0x7f3cd0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.708+0000 7f3ce52ba640 1 -- 172.21.15.72:0/4064479312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd0003520 con 0x7f3cd0005180 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.709+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cd0005180 0x7f3cd000b980 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3ce006c7f0 tx=0x7f3cd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87261e4992ba80f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.709+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd802cc80 con 0x7f3cd0005180 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.709+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd802cde0 con 0x7f3cd0005180 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.709+0000 7f3cdf7fe640 1 -- 172.21.15.72:0/4064479312 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8003bb0 con 0x7f3cd0005180 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.710+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/4064479312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cd0005180 msgr2=0x7f3cd000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.710+0000 7f3ce6d1d640 1 --2- 172.21.15.72:0/4064479312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cd0005180 0x7f3cd000b980 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3ce006c7f0 tx=0x7f3cd00057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.710+0000 7f3ce52ba640 1 -- 172.21.15.72:0/4064479312 reap_dead start 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.710+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/4064479312 shutdown_connections 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.710+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/4064479312 >> 172.21.15.72:0/4064479312 conn(0x7f3ce00b0a90 msgr2=0x7f3ce00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.710+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/4064479312 shutdown_connections 2023-12-12T09:54:36.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.710+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/4064479312 wait complete. 2023-12-12T09:54:36.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.712+0000 7f3ce6d1d640 1 Processor -- start 2023-12-12T09:54:36.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.712+0000 7f3ce6d1d640 1 -- start start 2023-12-12T09:54:36.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce6d1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd0005180 0x7f3ce015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:36.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce6d1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ce0140f40 0x7f3ce014adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:36.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce6d1d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ce0141ea0 0x7f3ce0142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:36.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce6d1d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce015ece0 con 0x7f3ce0141ea0 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce6d1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce015efc0 con 0x7f3ce0140f40 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce6d1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce015f2a0 con 0x7f3cd0005180 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce52ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ce0141ea0 0x7f3ce0142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce52ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ce0141ea0 0x7f3ce0142330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49838/0 (socket says 172.21.15.72:49838) 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce52ba640 1 -- 172.21.15.72:0/2167072633 learned_addr learned my addr 172.21.15.72:0/2167072633 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3ce4ab9640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd0005180 0x7f3ce015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.713+0000 7f3cdffff640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ce0140f40 0x7f3ce014adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce52ba640 1 -- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd0005180 msgr2=0x7f3ce015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd0005180 0x7f3ce015e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce52ba640 1 -- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ce0140f40 msgr2=0x7f3ce014adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ce0140f40 0x7f3ce014adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce52ba640 1 -- 172.21.15.72:0/2167072633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce015f580 con 0x7f3ce0141ea0 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce4ab9640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd0005180 0x7f3ce015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:36.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce52ba640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ce0141ea0 0x7f3ce0142330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3cd802bd30 tx=0x7f3cd8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:36.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3cdd7fa640 1 -- 172.21.15.72:0/2167072633 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd803c070 con 0x7f3ce0141ea0 2023-12-12T09:54:36.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3cdd7fa640 1 -- 172.21.15.72:0/2167072633 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8003e30 con 0x7f3ce0141ea0 2023-12-12T09:54:36.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce015f860 con 0x7f3ce0141ea0 2023-12-12T09:54:36.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3cdd7fa640 1 -- 172.21.15.72:0/2167072633 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8004730 con 0x7f3ce0141ea0 2023-12-12T09:54:36.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.714+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce014b300 con 0x7f3ce0141ea0 2023-12-12T09:54:36.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.715+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce00cb5c0 con 0x7f3ce0141ea0 2023-12-12T09:54:36.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.716+0000 7f3cdd7fa640 1 -- 172.21.15.72:0/2167072633 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3cd8037460 con 0x7f3ce0141ea0 2023-12-12T09:54:36.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.717+0000 7f3cdd7fa640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3cc406ae70 0x7f3cc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:36.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.717+0000 7f3cdd7fa640 1 -- 172.21.15.72:0/2167072633 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3cd80aadb0 con 0x7f3ce0141ea0 2023-12-12T09:54:36.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.717+0000 7f3ce4ab9640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3cc406ae70 0x7f3cc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:36.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.718+0000 7f3ce4ab9640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3cc406ae70 0x7f3cc406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4001bc0 tx=0x7f3cd4000f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:36.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.722+0000 7f3cdd7fa640 1 -- 172.21.15.72:0/2167072633 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3cd8075dc0 con 0x7f3ce0141ea0 2023-12-12T09:54:36.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.949+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ce00745d0 con 0x7f3ce0141ea0 2023-12-12T09:54:36.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.951+0000 7f3cdd7fa640 1 -- 172.21.15.72:0/2167072633 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f3cd8046090 con 0x7f3ce0141ea0 2023-12-12T09:54:36.953 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:36.953 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/130800926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":130800926},{"type":"v1","addr":"172.21.15.72:6836","nonce":130800926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/318582100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":318582100},{"type":"v1","addr":"172.21.15.72:6837","nonce":318582100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2380516950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2380516950},{"type":"v1","addr":"172.21.15.113:6835","nonce":2380516950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:54:32.882443+0000","modified":"2023-12-12T09:54:35.940697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/4181915308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4181915308},{"type":"v1","addr":"172.21.15.113:6834","nonce":4181915308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-12T09:54:36.953 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 25 2023-12-12T09:54:36.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3cc406ae70 msgr2=0x7f3cc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:36.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3cc406ae70 0x7f3cc406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4001bc0 tx=0x7f3cd4000f50 comp rx=0 tx=0).stop 2023-12-12T09:54:36.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ce0141ea0 msgr2=0x7f3ce0142330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:36.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ce0141ea0 0x7f3ce0142330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3cd802bd30 tx=0x7f3cd8004ab0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 shutdown_connections 2023-12-12T09:54:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3cc406ae70 0x7f3cc406d330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ce0141ea0 0x7f3ce0142330 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ce0140f40 0x7f3ce014adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 --2- 172.21.15.72:0/2167072633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cd0005180 0x7f3ce015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.953+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 >> 172.21.15.72:0/2167072633 conn(0x7f3ce00b0a90 msgr2=0x7f3ce00cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.954+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 shutdown_connections 2023-12-12T09:54:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:36.954+0000 7f3ce6d1d640 1 -- 172.21.15.72:0/2167072633 wait complete. 2023-12-12T09:54:36.973 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:37.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.187+0000 7f60ca133640 1 Processor -- start 2023-12-12T09:54:37.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.187+0000 7f60ca133640 1 -- start start 2023-12-12T09:54:37.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.188+0000 7f60ca133640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c4157a60 0x7f60c406c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.189+0000 7f60ca133640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60c406de20 0x7f60c415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.189+0000 7f60ca133640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c406ca40 con 0x7f60c415e0e0 2023-12-12T09:54:37.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.189+0000 7f60ca133640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c415e550 con 0x7f60c406de20 2023-12-12T09:54:37.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.189+0000 7f60ca133640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c4071cd0 con 0x7f60c4157a60 2023-12-12T09:54:37.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.189+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c4157a60 0x7f60c406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.189+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c4157a60 0x7f60c406c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:49498/0 (socket says 172.21.15.72:49498) 2023-12-12T09:54:37.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.189+0000 7f60c37fe640 1 -- 172.21.15.72:0/2974354481 learned_addr learned my addr 172.21.15.72:0/2974354481 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:37.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.189+0000 7f60c2ffd640 1 --2- 172.21.15.72:0/2974354481 >> v2:172.21.15.113:3300/0 conn(0x7f60c406de20 0x7f60c415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.190+0000 7f60c37fe640 1 -- 172.21.15.72:0/2974354481 >> v2:172.21.15.113:3300/0 conn(0x7f60c406de20 msgr2=0x7f60c415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:37.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.190+0000 7f60c37fe640 1 --2- 172.21.15.72:0/2974354481 >> v2:172.21.15.113:3300/0 conn(0x7f60c406de20 0x7f60c415dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.190+0000 7f60c37fe640 1 -- 172.21.15.72:0/2974354481 >> v1:172.21.15.113:6789/0 conn(0x7f60c415e0e0 legacy=0x7f60c415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.190+0000 7f60c37fe640 1 -- 172.21.15.72:0/2974354481 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c4070f10 con 0x7f60c4157a60 2023-12-12T09:54:37.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.191+0000 7f60c37fe640 1 --2- 172.21.15.72:0/2974354481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c4157a60 0x7f60c406c410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009870 tx=0x7f60b402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac9b9a89854a72d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:37.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.192+0000 7f60c27fc640 1 -- 172.21.15.72:0/2974354481 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4031640 con 0x7f60c4157a60 2023-12-12T09:54:37.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.192+0000 7f60c27fc640 1 -- 172.21.15.72:0/2974354481 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b40317a0 con 0x7f60c4157a60 2023-12-12T09:54:37.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.192+0000 7f60c27fc640 1 -- 172.21.15.72:0/2974354481 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4031b30 con 0x7f60c4157a60 2023-12-12T09:54:37.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.193+0000 7f60ca133640 1 -- 172.21.15.72:0/2974354481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c4157a60 msgr2=0x7f60c406c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.193+0000 7f60ca133640 1 --2- 172.21.15.72:0/2974354481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c4157a60 0x7f60c406c410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009870 tx=0x7f60b402b8f0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.193+0000 7f60ca133640 1 -- 172.21.15.72:0/2974354481 shutdown_connections 2023-12-12T09:54:37.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.193+0000 7f60ca133640 1 --2- 172.21.15.72:0/2974354481 >> v2:172.21.15.113:3300/0 conn(0x7f60c406de20 0x7f60c415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.193+0000 7f60ca133640 1 --2- 172.21.15.72:0/2974354481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c4157a60 0x7f60c406c410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.193+0000 7f60ca133640 1 -- 172.21.15.72:0/2974354481 >> 172.21.15.72:0/2974354481 conn(0x7f60c40b0a90 msgr2=0x7f60c40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:37.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.193+0000 7f60ca133640 1 -- 172.21.15.72:0/2974354481 shutdown_connections 2023-12-12T09:54:37.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.193+0000 7f60ca133640 1 -- 172.21.15.72:0/2974354481 wait complete. 2023-12-12T09:54:37.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.195+0000 7f60ca133640 1 Processor -- start 2023-12-12T09:54:37.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.195+0000 7f60ca133640 1 -- start start 2023-12-12T09:54:37.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60ca133640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c406de20 0x7f60c414a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60ca133640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60c4157a60 0x7f60c4158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60ca133640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c415e0e0 0x7f60c40701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60ca133640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c414adb0 con 0x7f60c4157a60 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60ca133640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41f3cc0 con 0x7f60c406de20 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60ca133640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c4159950 con 0x7f60c415e0e0 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60c2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60c4157a60 0x7f60c4158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60c2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60c4157a60 0x7f60c4158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49846/0 (socket says 172.21.15.72:49846) 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60c2ffd640 1 -- 172.21.15.72:0/3715732856 learned_addr learned my addr 172.21.15.72:0/3715732856 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60c3fff640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c415e0e0 0x7f60c40701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.196+0000 7f60c37fe640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c406de20 0x7f60c414a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60c2ffd640 1 -- 172.21.15.72:0/3715732856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c415e0e0 msgr2=0x7f60c40701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60c2ffd640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c415e0e0 0x7f60c40701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60c2ffd640 1 -- 172.21.15.72:0/3715732856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c406de20 msgr2=0x7f60c414a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60c2ffd640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c406de20 0x7f60c414a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60c2ffd640 1 -- 172.21.15.72:0/3715732856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40094d0 con 0x7f60c4157a60 2023-12-12T09:54:37.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60c2ffd640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60c4157a60 0x7f60c4158e90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e9b0 tx=0x7f60b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:37.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60abfff640 1 -- 172.21.15.72:0/3715732856 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8009670 con 0x7f60c4157a60 2023-12-12T09:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60abfff640 1 -- 172.21.15.72:0/3715732856 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b800c9a0 con 0x7f60c4157a60 2023-12-12T09:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60abfff640 1 -- 172.21.15.72:0/3715732856 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800cd50 con 0x7f60c4157a60 2023-12-12T09:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c41f3f80 con 0x7f60c4157a60 2023-12-12T09:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.197+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c41f4540 con 0x7f60c4157a60 2023-12-12T09:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.199+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c41f4700 con 0x7f60c4157a60 2023-12-12T09:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.200+0000 7f60abfff640 1 -- 172.21.15.72:0/3715732856 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f60b8013470 con 0x7f60c4157a60 2023-12-12T09:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.200+0000 7f60abfff640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60b006ae40 0x7f60b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.201+0000 7f60abfff640 1 -- 172.21.15.72:0/3715732856 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f60b80899a0 con 0x7f60c4157a60 2023-12-12T09:54:37.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.201+0000 7f60c37fe640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60b006ae40 0x7f60b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.201+0000 7f60c37fe640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60b006ae40 0x7f60b006d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60b4036000 tx=0x7f60b403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:37.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.205+0000 7f60abfff640 1 -- 172.21.15.72:0/3715732856 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f60b8054910 con 0x7f60c4157a60 2023-12-12T09:54:37.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.399+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60c40745d0 con 0x7f60c4157a60 2023-12-12T09:54:37.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.403+0000 7f60abfff640 1 -- 172.21.15.72:0/3715732856 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16774 (secure 0 0 0) 0x7f60b8017090 con 0x7f60c4157a60 2023-12-12T09:54:37.405 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:37.405 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":51,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:34.428514+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:54:30.694215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-12T09:54:31.894291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:37.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60b006ae40 msgr2=0x7f60b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60b006ae40 0x7f60b006d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60b4036000 tx=0x7f60b403c040 comp rx=0 tx=0).stop 2023-12-12T09:54:37.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60c4157a60 msgr2=0x7f60c4158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60c4157a60 0x7f60c4158e90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e9b0 tx=0x7f60b800ee70 comp rx=0 tx=0).stop 2023-12-12T09:54:37.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 shutdown_connections 2023-12-12T09:54:37.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60b006ae40 0x7f60b006d300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60c415e0e0 0x7f60c40701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60c4157a60 0x7f60c4158e90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 --2- 172.21.15.72:0/3715732856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60c406de20 0x7f60c414a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 >> 172.21.15.72:0/3715732856 conn(0x7f60c40b0a90 msgr2=0x7f60c40cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:37.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 shutdown_connections 2023-12-12T09:54:37.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.405+0000 7f60ca133640 1 -- 172.21.15.72:0/3715732856 wait complete. 2023-12-12T09:54:37.427 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-12T09:54:37.427 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:54:37.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.635+0000 7f7da759e640 1 Processor -- start 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.635+0000 7f7da759e640 1 -- start start 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da759e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 0x7f7da80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da759e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7da80d1b20 0x7f7da80b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da759e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da80b0a00 con 0x7f7da80d0c10 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da759e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da80ceea0 con 0x7f7da80d1b20 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da759e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da815d770 con 0x7f7da815d390 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da6d9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7da80d1b20 0x7f7da80b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da6d9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7da80d1b20 0x7f7da80b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49850/0 (socket says 172.21.15.72:49850) 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da6d9d640 1 -- 172.21.15.72:0/2277018227 learned_addr learned my addr 172.21.15.72:0/2277018227 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da659c640 1 --2- 172.21.15.72:0/2277018227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 0x7f7da80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da6d9d640 1 -- 172.21.15.72:0/2277018227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 msgr2=0x7f7da80b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.637+0000 7f7da6d9d640 1 --2- 172.21.15.72:0/2277018227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 0x7f7da80b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.638+0000 7f7da6d9d640 1 -- 172.21.15.72:0/2277018227 >> v1:172.21.15.113:6789/0 conn(0x7f7da80d0c10 legacy=0x7f7da80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.638+0000 7f7da6d9d640 1 -- 172.21.15.72:0/2277018227 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80b1840 con 0x7f7da80d1b20 2023-12-12T09:54:37.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.638+0000 7f7da659c640 1 --2- 172.21.15.72:0/2277018227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 0x7f7da80b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:37.641 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.639+0000 7f7da6d9d640 1 --2- 172.21.15.72:0/2277018227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d1b20 0x7f7da80b04c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009e50 tx=0x7f7da002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52628a503d475a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.639+0000 7f7da5d9b640 1 -- 172.21.15.72:0/2277018227 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da002c9e0 con 0x7f7da80d1b20 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.639+0000 7f7da5d9b640 1 -- 172.21.15.72:0/2277018227 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da002cb40 con 0x7f7da80d1b20 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.639+0000 7f7da5d9b640 1 -- 172.21.15.72:0/2277018227 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00343f0 con 0x7f7da80d1b20 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.640+0000 7f7da759e640 1 -- 172.21.15.72:0/2277018227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d1b20 msgr2=0x7f7da80b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.640+0000 7f7da759e640 1 --2- 172.21.15.72:0/2277018227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d1b20 0x7f7da80b04c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009e50 tx=0x7f7da002be40 comp rx=0 tx=0).stop 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.640+0000 7f7da759e640 1 -- 172.21.15.72:0/2277018227 shutdown_connections 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.640+0000 7f7da759e640 1 --2- 172.21.15.72:0/2277018227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d1b20 0x7f7da80b04c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.640+0000 7f7da759e640 1 --2- 172.21.15.72:0/2277018227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 0x7f7da80b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.640+0000 7f7da759e640 1 -- 172.21.15.72:0/2277018227 >> 172.21.15.72:0/2277018227 conn(0x7f7da80d2ec0 msgr2=0x7f7da80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.640+0000 7f7da759e640 1 -- 172.21.15.72:0/2277018227 shutdown_connections 2023-12-12T09:54:37.642 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.640+0000 7f7da759e640 1 -- 172.21.15.72:0/2277018227 wait complete. 2023-12-12T09:54:37.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.642+0000 7f7da759e640 1 Processor -- start 2023-12-12T09:54:37.643 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.642+0000 7f7da759e640 1 -- start start 2023-12-12T09:54:37.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da759e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d0c10 0x7f7da80cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da80d1b20 0x7f7da80b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da759e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 0x7f7da814bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da759e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b3db0 con 0x7f7da80d0c10 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da759e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b4090 con 0x7f7da80d1b20 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da759e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b3130 con 0x7f7da815d390 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d0c10 0x7f7da80cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da659c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d0c10 0x7f7da80cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49858/0 (socket says 172.21.15.72:49858) 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da659c640 1 -- 172.21.15.72:0/1599101717 learned_addr learned my addr 172.21.15.72:0/1599101717 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da659c640 1 -- 172.21.15.72:0/1599101717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 msgr2=0x7f7da814bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da659c640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 0x7f7da814bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da659c640 1 -- 172.21.15.72:0/1599101717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da80d1b20 msgr2=0x7f7da80b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da659c640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da80d1b20 0x7f7da80b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.643+0000 7f7da659c640 1 -- 172.21.15.72:0/1599101717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da0009b30 con 0x7f7da80d0c10 2023-12-12T09:54:37.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.644+0000 7f7da659c640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d0c10 0x7f7da80cdc70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c820 tx=0x7f7d9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:37.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.644+0000 7f7d9effd640 1 -- 172.21.15.72:0/1599101717 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98007a30 con 0x7f7da80d0c10 2023-12-12T09:54:37.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.644+0000 7f7d9effd640 1 -- 172.21.15.72:0/1599101717 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d98007b90 con 0x7f7da80d0c10 2023-12-12T09:54:37.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.644+0000 7f7d9effd640 1 -- 172.21.15.72:0/1599101717 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9800ce10 con 0x7f7da80d0c10 2023-12-12T09:54:37.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.644+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da80b3410 con 0x7f7da80d0c10 2023-12-12T09:54:37.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.644+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da80b39d0 con 0x7f7da80d0c10 2023-12-12T09:54:37.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.645+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da8074450 con 0x7f7da80d0c10 2023-12-12T09:54:37.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.646+0000 7f7d9effd640 1 -- 172.21.15.72:0/1599101717 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d980113f0 con 0x7f7da80d0c10 2023-12-12T09:54:37.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.647+0000 7f7d9effd640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d8806ae40 0x7f7d8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:37.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.647+0000 7f7d9effd640 1 -- 172.21.15.72:0/1599101717 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7d980898b0 con 0x7f7da80d0c10 2023-12-12T09:54:37.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.648+0000 7f7d9ffff640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d8806ae40 0x7f7d8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:37.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.648+0000 7f7d9ffff640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d8806ae40 0x7f7d8806d300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7da80ccdd0 tx=0x7f7d900079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:37.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.651+0000 7f7d9effd640 1 -- 172.21.15.72:0/1599101717 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d98054780 con 0x7f7da80d0c10 2023-12-12T09:54:37.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.837+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7da80b5470 con 0x7f7d8806ae40 2023-12-12T09:54:37.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.861+0000 7f7d9effd640 1 -- 172.21.15.72:0/1599101717 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286962 (secure 0 0 0) 0x7f7da80b5470 con 0x7f7d8806ae40 2023-12-12T09:54:37.868 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:37.869 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:54:37.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d8806ae40 msgr2=0x7f7d8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d8806ae40 0x7f7d8806d300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7da80ccdd0 tx=0x7f7d900079b0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d0c10 msgr2=0x7f7da80cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:37.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d0c10 0x7f7da80cdc70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c820 tx=0x7f7d9800cce0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 shutdown_connections 2023-12-12T09:54:37.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f7d8806ae40 0x7f7d8806d300 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7da815d390 0x7f7da814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da80d1b20 0x7f7da80b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 --2- 172.21.15.72:0/1599101717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7da80d0c10 0x7f7da80cdc70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:37.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 >> 172.21.15.72:0/1599101717 conn(0x7f7da80d2ec0 msgr2=0x7f7da80c9350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:37.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 shutdown_connections 2023-12-12T09:54:37.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:37.868+0000 7f7da759e640 1 -- 172.21.15.72:0/1599101717 wait complete. 2023-12-12T09:54:37.889 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-12-12T09:54:37.593148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255364,"kb_used_data":5888,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719356,"statfs":{"total":773094113280,"available":772832620544,"internally_reserved":0,"allocated":6029312,"data_stored":3292183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001616"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.446501+0000","last_change":"2023-12-12T09:54:33.435866+0000","last_active":"2023-12-12T09:54:33.446501+0000","last_peered":"2023-12-12T09:54:33.446501+0000","last_clean":"2023-12-12T09:54:33.446501+0000","last_became_active":"2023-12-12T09:54:33.435542+0000","last_became_peered":"2023-12-12T09:54:33.435542+0000","last_unstale":"2023-12-12T09:54:33.446501+0000","last_undegraded":"2023-12-12T09:54:33.446501+0000","last_fullsized":"2023-12-12T09:54:33.446501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:56.972947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842499+0000","last_change":"2023-12-12T09:54:33.454431+0000","last_active":"2023-12-12T09:54:34.842499+0000","last_peered":"2023-12-12T09:54:34.842499+0000","last_clean":"2023-12-12T09:54:34.842499+0000","last_became_active":"2023-12-12T09:54:33.454105+0000","last_became_peered":"2023-12-12T09:54:33.454105+0000","last_unstale":"2023-12-12T09:54:34.842499+0000","last_undegraded":"2023-12-12T09:54:34.842499+0000","last_fullsized":"2023-12-12T09:54:34.842499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:41:48.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.458261+0000","last_change":"2023-12-12T09:54:33.441791+0000","last_active":"2023-12-12T09:54:33.458261+0000","last_peered":"2023-12-12T09:54:33.458261+0000","last_clean":"2023-12-12T09:54:33.458261+0000","last_became_active":"2023-12-12T09:54:33.441528+0000","last_became_peered":"2023-12-12T09:54:33.441528+0000","last_unstale":"2023-12-12T09:54:33.458261+0000","last_undegraded":"2023-12-12T09:54:33.458261+0000","last_fullsized":"2023-12-12T09:54:33.458261+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:24.157458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842541+0000","last_change":"2023-12-12T09:54:33.456039+0000","last_active":"2023-12-12T09:54:34.842541+0000","last_peered":"2023-12-12T09:54:34.842541+0000","last_clean":"2023-12-12T09:54:34.842541+0000","last_became_active":"2023-12-12T09:54:33.455729+0000","last_became_peered":"2023-12-12T09:54:33.455729+0000","last_unstale":"2023-12-12T09:54:34.842541+0000","last_undegraded":"2023-12-12T09:54:34.842541+0000","last_fullsized":"2023-12-12T09:54:34.842541+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:14:27.591850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.454004+0000","last_change":"2023-12-12T09:54:33.446247+0000","last_active":"2023-12-12T09:54:33.454004+0000","last_peered":"2023-12-12T09:54:33.454004+0000","last_clean":"2023-12-12T09:54:33.454004+0000","last_became_active":"2023-12-12T09:54:33.445978+0000","last_became_peered":"2023-12-12T09:54:33.445978+0000","last_unstale":"2023-12-12T09:54:33.454004+0000","last_undegraded":"2023-12-12T09:54:33.454004+0000","last_fullsized":"2023-12-12T09:54:33.454004+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:47.433030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449496+0000","last_change":"2023-12-12T09:54:33.444230+0000","last_active":"2023-12-12T09:54:33.449496+0000","last_peered":"2023-12-12T09:54:33.449496+0000","last_clean":"2023-12-12T09:54:33.449496+0000","last_became_active":"2023-12-12T09:54:33.443949+0000","last_became_peered":"2023-12-12T09:54:33.443949+0000","last_unstale":"2023-12-12T09:54:33.449496+0000","last_undegraded":"2023-12-12T09:54:33.449496+0000","last_fullsized":"2023-12-12T09:54:33.449496+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:47.433030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463641+0000","last_change":"2023-12-12T09:54:33.453152+0000","last_active":"2023-12-12T09:54:33.463641+0000","last_peered":"2023-12-12T09:54:33.463641+0000","last_clean":"2023-12-12T09:54:33.463641+0000","last_became_active":"2023-12-12T09:54:33.447789+0000","last_became_peered":"2023-12-12T09:54:33.447789+0000","last_unstale":"2023-12-12T09:54:33.463641+0000","last_undegraded":"2023-12-12T09:54:33.463641+0000","last_fullsized":"2023-12-12T09:54:33.463641+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:22:11.410125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.465627+0000","last_change":"2023-12-12T09:54:33.463606+0000","last_active":"2023-12-12T09:54:33.465627+0000","last_peered":"2023-12-12T09:54:33.465627+0000","last_clean":"2023-12-12T09:54:33.465627+0000","last_became_active":"2023-12-12T09:54:33.463339+0000","last_became_peered":"2023-12-12T09:54:33.463339+0000","last_unstale":"2023-12-12T09:54:33.465627+0000","last_undegraded":"2023-12-12T09:54:33.465627+0000","last_fullsized":"2023-12-12T09:54:33.465627+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:45:13.037742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.465638+0000","last_change":"2023-12-12T09:54:33.463635+0000","last_active":"2023-12-12T09:54:33.465638+0000","last_peered":"2023-12-12T09:54:33.465638+0000","last_clean":"2023-12-12T09:54:33.465638+0000","last_became_active":"2023-12-12T09:54:33.463399+0000","last_became_peered":"2023-12-12T09:54:33.463399+0000","last_unstale":"2023-12-12T09:54:33.465638+0000","last_undegraded":"2023-12-12T09:54:33.465638+0000","last_fullsized":"2023-12-12T09:54:33.465638+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:05:19.068458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.459062+0000","last_change":"2023-12-12T09:54:33.455702+0000","last_active":"2023-12-12T09:54:33.459062+0000","last_peered":"2023-12-12T09:54:33.459062+0000","last_clean":"2023-12-12T09:54:33.459062+0000","last_became_active":"2023-12-12T09:54:33.453529+0000","last_became_peered":"2023-12-12T09:54:33.453529+0000","last_unstale":"2023-12-12T09:54:33.459062+0000","last_undegraded":"2023-12-12T09:54:33.459062+0000","last_fullsized":"2023-12-12T09:54:33.459062+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:01:17.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.446639+0000","last_change":"2023-12-12T09:54:33.443656+0000","last_active":"2023-12-12T09:54:34.446639+0000","last_peered":"2023-12-12T09:54:34.446639+0000","last_clean":"2023-12-12T09:54:34.446639+0000","last_became_active":"2023-12-12T09:54:33.443318+0000","last_became_peered":"2023-12-12T09:54:33.443318+0000","last_unstale":"2023-12-12T09:54:34.446639+0000","last_undegraded":"2023-12-12T09:54:34.446639+0000","last_fullsized":"2023-12-12T09:54:34.446639+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:07:19.412716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451117+0000","last_change":"2023-12-12T09:54:33.444827+0000","last_active":"2023-12-12T09:54:33.451117+0000","last_peered":"2023-12-12T09:54:33.451117+0000","last_clean":"2023-12-12T09:54:33.451117+0000","last_became_active":"2023-12-12T09:54:33.444497+0000","last_became_peered":"2023-12-12T09:54:33.444497+0000","last_unstale":"2023-12-12T09:54:33.451117+0000","last_undegraded":"2023-12-12T09:54:33.451117+0000","last_fullsized":"2023-12-12T09:54:33.451117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:27:59.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.459070+0000","last_change":"2023-12-12T09:54:33.456728+0000","last_active":"2023-12-12T09:54:33.459070+0000","last_peered":"2023-12-12T09:54:33.459070+0000","last_clean":"2023-12-12T09:54:33.459070+0000","last_became_active":"2023-12-12T09:54:33.456383+0000","last_became_peered":"2023-12-12T09:54:33.456383+0000","last_unstale":"2023-12-12T09:54:33.459070+0000","last_undegraded":"2023-12-12T09:54:33.459070+0000","last_fullsized":"2023-12-12T09:54:33.459070+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:22:11.410125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453775+0000","last_change":"2023-12-12T09:54:33.442998+0000","last_active":"2023-12-12T09:54:33.453775+0000","last_peered":"2023-12-12T09:54:33.453775+0000","last_clean":"2023-12-12T09:54:33.453775+0000","last_became_active":"2023-12-12T09:54:33.442319+0000","last_became_peered":"2023-12-12T09:54:33.442319+0000","last_unstale":"2023-12-12T09:54:33.453775+0000","last_undegraded":"2023-12-12T09:54:33.453775+0000","last_fullsized":"2023-12-12T09:54:33.453775+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:12:49.054669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.841968+0000","last_change":"2023-12-12T09:54:33.449273+0000","last_active":"2023-12-12T09:54:34.841968+0000","last_peered":"2023-12-12T09:54:34.841968+0000","last_clean":"2023-12-12T09:54:34.841968+0000","last_became_active":"2023-12-12T09:54:33.448949+0000","last_became_peered":"2023-12-12T09:54:33.448949+0000","last_unstale":"2023-12-12T09:54:34.841968+0000","last_undegraded":"2023-12-12T09:54:34.841968+0000","last_fullsized":"2023-12-12T09:54:34.841968+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:03.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449031+0000","last_change":"2023-12-12T09:54:33.444507+0000","last_active":"2023-12-12T09:54:33.449031+0000","last_peered":"2023-12-12T09:54:33.449031+0000","last_clean":"2023-12-12T09:54:33.449031+0000","last_became_active":"2023-12-12T09:54:33.444256+0000","last_became_peered":"2023-12-12T09:54:33.444256+0000","last_unstale":"2023-12-12T09:54:33.449031+0000","last_undegraded":"2023-12-12T09:54:33.449031+0000","last_fullsized":"2023-12-12T09:54:33.449031+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:30:50.396953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463749+0000","last_change":"2023-12-12T09:54:33.459282+0000","last_active":"2023-12-12T09:54:33.463749+0000","last_peered":"2023-12-12T09:54:33.463749+0000","last_clean":"2023-12-12T09:54:33.463749+0000","last_became_active":"2023-12-12T09:54:33.449272+0000","last_became_peered":"2023-12-12T09:54:33.449272+0000","last_unstale":"2023-12-12T09:54:33.463749+0000","last_undegraded":"2023-12-12T09:54:33.463749+0000","last_fullsized":"2023-12-12T09:54:33.463749+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:10:40.233777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842683+0000","last_change":"2023-12-12T09:54:33.454480+0000","last_active":"2023-12-12T09:54:34.842683+0000","last_peered":"2023-12-12T09:54:34.842683+0000","last_clean":"2023-12-12T09:54:34.842683+0000","last_became_active":"2023-12-12T09:54:33.454194+0000","last_became_peered":"2023-12-12T09:54:33.454194+0000","last_unstale":"2023-12-12T09:54:34.842683+0000","last_undegraded":"2023-12-12T09:54:34.842683+0000","last_fullsized":"2023-12-12T09:54:34.842683+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:44:17.389128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463632+0000","last_change":"2023-12-12T09:54:33.461140+0000","last_active":"2023-12-12T09:54:33.463632+0000","last_peered":"2023-12-12T09:54:33.463632+0000","last_clean":"2023-12-12T09:54:33.463632+0000","last_became_active":"2023-12-12T09:54:33.460869+0000","last_became_peered":"2023-12-12T09:54:33.460869+0000","last_unstale":"2023-12-12T09:54:33.463632+0000","last_undegraded":"2023-12-12T09:54:33.463632+0000","last_fullsized":"2023-12-12T09:54:33.463632+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:27:59.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451947+0000","last_change":"2023-12-12T09:54:33.444494+0000","last_active":"2023-12-12T09:54:33.451947+0000","last_peered":"2023-12-12T09:54:33.451947+0000","last_clean":"2023-12-12T09:54:33.451947+0000","last_became_active":"2023-12-12T09:54:33.444120+0000","last_became_peered":"2023-12-12T09:54:33.444120+0000","last_unstale":"2023-12-12T09:54:33.451947+0000","last_undegraded":"2023-12-12T09:54:33.451947+0000","last_fullsized":"2023-12-12T09:54:33.451947+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:18:52.128679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.445509+0000","last_change":"2023-12-12T09:54:33.442029+0000","last_active":"2023-12-12T09:54:33.445509+0000","last_peered":"2023-12-12T09:54:33.445509+0000","last_clean":"2023-12-12T09:54:33.445509+0000","last_became_active":"2023-12-12T09:54:33.441737+0000","last_became_peered":"2023-12-12T09:54:33.441737+0000","last_unstale":"2023-12-12T09:54:33.445509+0000","last_undegraded":"2023-12-12T09:54:33.445509+0000","last_fullsized":"2023-12-12T09:54:33.445509+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:01:17.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463428+0000","last_change":"2023-12-12T09:54:33.462336+0000","last_active":"2023-12-12T09:54:33.463428+0000","last_peered":"2023-12-12T09:54:33.463428+0000","last_clean":"2023-12-12T09:54:33.463428+0000","last_became_active":"2023-12-12T09:54:33.462056+0000","last_became_peered":"2023-12-12T09:54:33.462056+0000","last_unstale":"2023-12-12T09:54:33.463428+0000","last_undegraded":"2023-12-12T09:54:33.463428+0000","last_fullsized":"2023-12-12T09:54:33.463428+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:18:15.115398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.445833+0000","last_change":"2023-12-12T09:54:33.440468+0000","last_active":"2023-12-12T09:54:34.445833+0000","last_peered":"2023-12-12T09:54:34.445833+0000","last_clean":"2023-12-12T09:54:34.445833+0000","last_became_active":"2023-12-12T09:54:33.440190+0000","last_became_peered":"2023-12-12T09:54:33.440190+0000","last_unstale":"2023-12-12T09:54:34.445833+0000","last_undegraded":"2023-12-12T09:54:34.445833+0000","last_fullsized":"2023-12-12T09:54:34.445833+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:55:48.144356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463027+0000","last_change":"2023-12-12T09:54:33.452024+0000","last_active":"2023-12-12T09:54:33.463027+0000","last_peered":"2023-12-12T09:54:33.463027+0000","last_clean":"2023-12-12T09:54:33.463027+0000","last_became_active":"2023-12-12T09:54:33.451593+0000","last_became_peered":"2023-12-12T09:54:33.451593+0000","last_unstale":"2023-12-12T09:54:33.463027+0000","last_undegraded":"2023-12-12T09:54:33.463027+0000","last_fullsized":"2023-12-12T09:54:33.463027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:41.494785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463768+0000","last_change":"2023-12-12T09:54:33.459917+0000","last_active":"2023-12-12T09:54:33.463768+0000","last_peered":"2023-12-12T09:54:33.463768+0000","last_clean":"2023-12-12T09:54:33.463768+0000","last_became_active":"2023-12-12T09:54:33.459595+0000","last_became_peered":"2023-12-12T09:54:33.459595+0000","last_unstale":"2023-12-12T09:54:33.463768+0000","last_undegraded":"2023-12-12T09:54:33.463768+0000","last_fullsized":"2023-12-12T09:54:33.463768+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:54:26.808349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453906+0000","last_change":"2023-12-12T09:54:33.451525+0000","last_active":"2023-12-12T09:54:33.453906+0000","last_peered":"2023-12-12T09:54:33.453906+0000","last_clean":"2023-12-12T09:54:33.453906+0000","last_became_active":"2023-12-12T09:54:33.448138+0000","last_became_peered":"2023-12-12T09:54:33.448138+0000","last_unstale":"2023-12-12T09:54:33.453906+0000","last_undegraded":"2023-12-12T09:54:33.453906+0000","last_fullsized":"2023-12-12T09:54:33.453906+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:14:27.591850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.460231+0000","last_change":"2023-12-12T09:54:33.458831+0000","last_active":"2023-12-12T09:54:33.460231+0000","last_peered":"2023-12-12T09:54:33.460231+0000","last_clean":"2023-12-12T09:54:33.460231+0000","last_became_active":"2023-12-12T09:54:33.458379+0000","last_became_peered":"2023-12-12T09:54:33.458379+0000","last_unstale":"2023-12-12T09:54:33.460231+0000","last_undegraded":"2023-12-12T09:54:33.460231+0000","last_fullsized":"2023-12-12T09:54:33.460231+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:40:29.637509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.438704+0000","last_change":"2023-12-12T09:54:33.436244+0000","last_active":"2023-12-12T09:54:33.438704+0000","last_peered":"2023-12-12T09:54:33.438704+0000","last_clean":"2023-12-12T09:54:33.438704+0000","last_became_active":"2023-12-12T09:54:33.435900+0000","last_became_peered":"2023-12-12T09:54:33.435900+0000","last_unstale":"2023-12-12T09:54:33.438704+0000","last_undegraded":"2023-12-12T09:54:33.438704+0000","last_fullsized":"2023-12-12T09:54:33.438704+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:32.422515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449359+0000","last_change":"2023-12-12T09:54:33.440417+0000","last_active":"2023-12-12T09:54:33.449359+0000","last_peered":"2023-12-12T09:54:33.449359+0000","last_clean":"2023-12-12T09:54:33.449359+0000","last_became_active":"2023-12-12T09:54:33.440149+0000","last_became_peered":"2023-12-12T09:54:33.440149+0000","last_unstale":"2023-12-12T09:54:33.449359+0000","last_undegraded":"2023-12-12T09:54:33.449359+0000","last_fullsized":"2023-12-12T09:54:33.449359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:41:48.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842394+0000","last_change":"2023-12-12T09:54:33.453918+0000","last_active":"2023-12-12T09:54:34.842394+0000","last_peered":"2023-12-12T09:54:34.842394+0000","last_clean":"2023-12-12T09:54:34.842394+0000","last_became_active":"2023-12-12T09:54:33.452501+0000","last_became_peered":"2023-12-12T09:54:33.452501+0000","last_unstale":"2023-12-12T09:54:34.842394+0000","last_undegraded":"2023-12-12T09:54:34.842394+0000","last_fullsized":"2023-12-12T09:54:34.842394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:45:13.037742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.460762+0000","last_change":"2023-12-12T09:54:33.445247+0000","last_active":"2023-12-12T09:54:33.460762+0000","last_peered":"2023-12-12T09:54:33.460762+0000","last_clean":"2023-12-12T09:54:33.460762+0000","last_became_active":"2023-12-12T09:54:33.443723+0000","last_became_peered":"2023-12-12T09:54:33.443723+0000","last_unstale":"2023-12-12T09:54:33.460762+0000","last_undegraded":"2023-12-12T09:54:33.460762+0000","last_fullsized":"2023-12-12T09:54:33.460762+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:33:25.127743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448289+0000","last_change":"2023-12-12T09:54:33.453301+0000","last_active":"2023-12-12T09:54:34.448289+0000","last_peered":"2023-12-12T09:54:34.448289+0000","last_clean":"2023-12-12T09:54:34.448289+0000","last_became_active":"2023-12-12T09:54:33.452939+0000","last_became_peered":"2023-12-12T09:54:33.452939+0000","last_unstale":"2023-12-12T09:54:34.448289+0000","last_undegraded":"2023-12-12T09:54:34.448289+0000","last_fullsized":"2023-12-12T09:54:34.448289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:03.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.841387+0000","last_change":"2023-12-12T09:54:33.452118+0000","last_active":"2023-12-12T09:54:34.841387+0000","last_peered":"2023-12-12T09:54:34.841387+0000","last_clean":"2023-12-12T09:54:34.841387+0000","last_became_active":"2023-12-12T09:54:33.451052+0000","last_became_peered":"2023-12-12T09:54:33.451052+0000","last_unstale":"2023-12-12T09:54:34.841387+0000","last_undegraded":"2023-12-12T09:54:34.841387+0000","last_fullsized":"2023-12-12T09:54:34.841387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:04:17.483412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.446047+0000","last_change":"2023-12-12T09:54:32.442157+0000","last_active":"2023-12-12T09:54:33.446047+0000","last_peered":"2023-12-12T09:54:33.446047+0000","last_clean":"2023-12-12T09:54:33.446047+0000","last_became_active":"2023-12-12T09:54:32.441813+0000","last_became_peered":"2023-12-12T09:54:32.441813+0000","last_unstale":"2023-12-12T09:54:33.446047+0000","last_undegraded":"2023-12-12T09:54:33.446047+0000","last_fullsized":"2023-12-12T09:54:33.446047+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:40:28.622399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448714+0000","last_change":"2023-12-12T09:54:33.447006+0000","last_active":"2023-12-12T09:54:34.448714+0000","last_peered":"2023-12-12T09:54:34.448714+0000","last_clean":"2023-12-12T09:54:34.448714+0000","last_became_active":"2023-12-12T09:54:33.444075+0000","last_became_peered":"2023-12-12T09:54:33.444075+0000","last_unstale":"2023-12-12T09:54:34.448714+0000","last_undegraded":"2023-12-12T09:54:34.448714+0000","last_fullsized":"2023-12-12T09:54:34.448714+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:45:13.037742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.457607+0000","last_change":"2023-12-12T09:54:32.448536+0000","last_active":"2023-12-12T09:54:33.457607+0000","last_peered":"2023-12-12T09:54:33.457607+0000","last_clean":"2023-12-12T09:54:33.457607+0000","last_became_active":"2023-12-12T09:54:32.448117+0000","last_became_peered":"2023-12-12T09:54:32.448117+0000","last_unstale":"2023-12-12T09:54:33.457607+0000","last_undegraded":"2023-12-12T09:54:33.457607+0000","last_fullsized":"2023-12-12T09:54:33.457607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:33:55.675558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451798+0000","last_change":"2023-12-12T09:54:33.444330+0000","last_active":"2023-12-12T09:54:33.451798+0000","last_peered":"2023-12-12T09:54:33.451798+0000","last_clean":"2023-12-12T09:54:33.451798+0000","last_became_active":"2023-12-12T09:54:33.444016+0000","last_became_peered":"2023-12-12T09:54:33.444016+0000","last_unstale":"2023-12-12T09:54:33.451798+0000","last_undegraded":"2023-12-12T09:54:33.451798+0000","last_fullsized":"2023-12-12T09:54:33.451798+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:47.433030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451068+0000","last_change":"2023-12-12T09:54:32.446663+0000","last_active":"2023-12-12T09:54:33.451068+0000","last_peered":"2023-12-12T09:54:33.451068+0000","last_clean":"2023-12-12T09:54:33.451068+0000","last_became_active":"2023-12-12T09:54:32.445288+0000","last_became_peered":"2023-12-12T09:54:32.445288+0000","last_unstale":"2023-12-12T09:54:33.451068+0000","last_undegraded":"2023-12-12T09:54:33.451068+0000","last_fullsized":"2023-12-12T09:54:33.451068+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:33:24.112633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453812+0000","last_change":"2023-12-12T09:54:33.450915+0000","last_active":"2023-12-12T09:54:33.453812+0000","last_peered":"2023-12-12T09:54:33.453812+0000","last_clean":"2023-12-12T09:54:33.453812+0000","last_became_active":"2023-12-12T09:54:33.450523+0000","last_became_peered":"2023-12-12T09:54:33.450523+0000","last_unstale":"2023-12-12T09:54:33.453812+0000","last_undegraded":"2023-12-12T09:54:33.453812+0000","last_fullsized":"2023-12-12T09:54:33.453812+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:04:17.483412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.449520+0000","last_change":"2023-12-12T09:54:32.445526+0000","last_active":"2023-12-12T09:54:34.449520+0000","last_peered":"2023-12-12T09:54:34.449520+0000","last_clean":"2023-12-12T09:54:34.449520+0000","last_became_active":"2023-12-12T09:54:32.445271+0000","last_became_peered":"2023-12-12T09:54:32.445271+0000","last_unstale":"2023-12-12T09:54:34.449520+0000","last_undegraded":"2023-12-12T09:54:34.449520+0000","last_fullsized":"2023-12-12T09:54:34.449520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:18:14.100288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.462503+0000","last_change":"2023-12-12T09:54:33.461071+0000","last_active":"2023-12-12T09:54:33.462503+0000","last_peered":"2023-12-12T09:54:33.462503+0000","last_clean":"2023-12-12T09:54:33.462503+0000","last_became_active":"2023-12-12T09:54:33.460803+0000","last_became_peered":"2023-12-12T09:54:33.460803+0000","last_unstale":"2023-12-12T09:54:33.462503+0000","last_undegraded":"2023-12-12T09:54:33.462503+0000","last_fullsized":"2023-12-12T09:54:33.462503+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:56.972947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.460152+0000","last_change":"2023-12-12T09:54:32.431758+0000","last_active":"2023-12-12T09:54:33.460152+0000","last_peered":"2023-12-12T09:54:33.460152+0000","last_clean":"2023-12-12T09:54:33.460152+0000","last_became_active":"2023-12-12T09:54:32.431479+0000","last_became_peered":"2023-12-12T09:54:32.431479+0000","last_unstale":"2023-12-12T09:54:33.460152+0000","last_undegraded":"2023-12-12T09:54:33.460152+0000","last_fullsized":"2023-12-12T09:54:33.460152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:14:26.576740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842370+0000","last_change":"2023-12-12T09:54:33.457942+0000","last_active":"2023-12-12T09:54:34.842370+0000","last_peered":"2023-12-12T09:54:34.842370+0000","last_clean":"2023-12-12T09:54:34.842370+0000","last_became_active":"2023-12-12T09:54:33.457687+0000","last_became_peered":"2023-12-12T09:54:33.457687+0000","last_unstale":"2023-12-12T09:54:34.842370+0000","last_undegraded":"2023-12-12T09:54:34.842370+0000","last_fullsized":"2023-12-12T09:54:34.842370+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:10:40.233777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.443151+0000","last_change":"2023-12-12T09:54:32.441501+0000","last_active":"2023-12-12T09:54:33.443151+0000","last_peered":"2023-12-12T09:54:33.443151+0000","last_clean":"2023-12-12T09:54:33.443151+0000","last_became_active":"2023-12-12T09:54:32.441140+0000","last_became_peered":"2023-12-12T09:54:32.441140+0000","last_unstale":"2023-12-12T09:54:33.443151+0000","last_undegraded":"2023-12-12T09:54:33.443151+0000","last_fullsized":"2023-12-12T09:54:33.443151+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:20.188945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451627+0000","last_change":"2023-12-12T09:54:33.447877+0000","last_active":"2023-12-12T09:54:33.451627+0000","last_peered":"2023-12-12T09:54:33.451627+0000","last_clean":"2023-12-12T09:54:33.451627+0000","last_became_active":"2023-12-12T09:54:33.447564+0000","last_became_peered":"2023-12-12T09:54:33.447564+0000","last_unstale":"2023-12-12T09:54:33.451627+0000","last_undegraded":"2023-12-12T09:54:33.451627+0000","last_fullsized":"2023-12-12T09:54:33.451627+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:24.157458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.425593+0000","last_change":"2023-12-12T09:54:32.440483+0000","last_active":"2023-12-12T09:54:33.425593+0000","last_peered":"2023-12-12T09:54:33.425593+0000","last_clean":"2023-12-12T09:54:33.425593+0000","last_became_active":"2023-12-12T09:54:32.440185+0000","last_became_peered":"2023-12-12T09:54:32.440185+0000","last_unstale":"2023-12-12T09:54:33.425593+0000","last_undegraded":"2023-12-12T09:54:33.425593+0000","last_fullsized":"2023-12-12T09:54:33.425593+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:55.957837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842423+0000","last_change":"2023-12-12T09:54:33.460290+0000","last_active":"2023-12-12T09:54:34.842423+0000","last_peered":"2023-12-12T09:54:34.842423+0000","last_clean":"2023-12-12T09:54:34.842423+0000","last_became_active":"2023-12-12T09:54:33.460020+0000","last_became_peered":"2023-12-12T09:54:33.460020+0000","last_unstale":"2023-12-12T09:54:34.842423+0000","last_undegraded":"2023-12-12T09:54:34.842423+0000","last_fullsized":"2023-12-12T09:54:34.842423+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:54.293178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.425722+0000","last_change":"2023-12-12T09:54:32.440839+0000","last_active":"2023-12-12T09:54:33.425722+0000","last_peered":"2023-12-12T09:54:33.425722+0000","last_clean":"2023-12-12T09:54:33.425722+0000","last_became_active":"2023-12-12T09:54:32.440502+0000","last_became_peered":"2023-12-12T09:54:32.440502+0000","last_unstale":"2023-12-12T09:54:33.425722+0000","last_undegraded":"2023-12-12T09:54:33.425722+0000","last_fullsized":"2023-12-12T09:54:33.425722+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:35.059090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.454421+0000","last_change":"2023-12-12T09:54:33.450701+0000","last_active":"2023-12-12T09:54:33.454421+0000","last_peered":"2023-12-12T09:54:33.454421+0000","last_clean":"2023-12-12T09:54:33.454421+0000","last_became_active":"2023-12-12T09:54:33.450346+0000","last_became_peered":"2023-12-12T09:54:33.450346+0000","last_unstale":"2023-12-12T09:54:33.454421+0000","last_undegraded":"2023-12-12T09:54:33.454421+0000","last_fullsized":"2023-12-12T09:54:33.454421+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:18:52.128679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842631+0000","last_change":"2023-12-12T09:54:32.440358+0000","last_active":"2023-12-12T09:54:34.842631+0000","last_peered":"2023-12-12T09:54:34.842631+0000","last_clean":"2023-12-12T09:54:34.842631+0000","last_became_active":"2023-12-12T09:54:32.439955+0000","last_became_peered":"2023-12-12T09:54:32.439955+0000","last_unstale":"2023-12-12T09:54:34.842631+0000","last_undegraded":"2023-12-12T09:54:34.842631+0000","last_fullsized":"2023-12-12T09:54:34.842631+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:27:58.029668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448282+0000","last_change":"2023-12-12T09:54:33.448989+0000","last_active":"2023-12-12T09:54:34.448282+0000","last_peered":"2023-12-12T09:54:34.448282+0000","last_clean":"2023-12-12T09:54:34.448282+0000","last_became_active":"2023-12-12T09:54:33.448732+0000","last_became_peered":"2023-12-12T09:54:33.448732+0000","last_unstale":"2023-12-12T09:54:34.448282+0000","last_undegraded":"2023-12-12T09:54:34.448282+0000","last_fullsized":"2023-12-12T09:54:34.448282+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:04:17.483412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.432144+0000","last_change":"2023-12-12T09:54:32.440990+0000","last_active":"2023-12-12T09:54:33.432144+0000","last_peered":"2023-12-12T09:54:33.432144+0000","last_clean":"2023-12-12T09:54:33.432144+0000","last_became_active":"2023-12-12T09:54:32.440702+0000","last_became_peered":"2023-12-12T09:54:32.440702+0000","last_unstale":"2023-12-12T09:54:33.432144+0000","last_undegraded":"2023-12-12T09:54:33.432144+0000","last_fullsized":"2023-12-12T09:54:33.432144+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:31.407405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449821+0000","last_change":"2023-12-12T09:54:33.436001+0000","last_active":"2023-12-12T09:54:33.449821+0000","last_peered":"2023-12-12T09:54:33.449821+0000","last_clean":"2023-12-12T09:54:33.449821+0000","last_became_active":"2023-12-12T09:54:33.434079+0000","last_became_peered":"2023-12-12T09:54:33.434079+0000","last_unstale":"2023-12-12T09:54:33.449821+0000","last_undegraded":"2023-12-12T09:54:33.449821+0000","last_fullsized":"2023-12-12T09:54:33.449821+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:33:56.690667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842532+0000","last_change":"2023-12-12T09:54:32.443678+0000","last_active":"2023-12-12T09:54:34.842532+0000","last_peered":"2023-12-12T09:54:34.842532+0000","last_clean":"2023-12-12T09:54:34.842532+0000","last_became_active":"2023-12-12T09:54:32.443383+0000","last_became_peered":"2023-12-12T09:54:32.443383+0000","last_unstale":"2023-12-12T09:54:34.842532+0000","last_undegraded":"2023-12-12T09:54:34.842532+0000","last_fullsized":"2023-12-12T09:54:34.842532+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:33:55.675558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.454024+0000","last_change":"2023-12-12T09:54:33.448194+0000","last_active":"2023-12-12T09:54:33.454024+0000","last_peered":"2023-12-12T09:54:33.454024+0000","last_clean":"2023-12-12T09:54:33.454024+0000","last_became_active":"2023-12-12T09:54:33.447874+0000","last_became_peered":"2023-12-12T09:54:33.447874+0000","last_unstale":"2023-12-12T09:54:33.454024+0000","last_undegraded":"2023-12-12T09:54:33.454024+0000","last_fullsized":"2023-12-12T09:54:33.454024+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:28:27.268634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.466039+0000","last_change":"2023-12-12T09:54:32.444522+0000","last_active":"2023-12-12T09:54:33.466039+0000","last_peered":"2023-12-12T09:54:33.466039+0000","last_clean":"2023-12-12T09:54:33.466039+0000","last_became_active":"2023-12-12T09:54:32.444253+0000","last_became_peered":"2023-12-12T09:54:32.444253+0000","last_unstale":"2023-12-12T09:54:33.466039+0000","last_undegraded":"2023-12-12T09:54:33.466039+0000","last_fullsized":"2023-12-12T09:54:33.466039+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:43:03.013525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448716+0000","last_change":"2023-12-12T09:54:33.445430+0000","last_active":"2023-12-12T09:54:34.448716+0000","last_peered":"2023-12-12T09:54:34.448716+0000","last_clean":"2023-12-12T09:54:34.448716+0000","last_became_active":"2023-12-12T09:54:33.445063+0000","last_became_peered":"2023-12-12T09:54:33.445063+0000","last_unstale":"2023-12-12T09:54:34.448716+0000","last_undegraded":"2023-12-12T09:54:34.448716+0000","last_fullsized":"2023-12-12T09:54:34.448716+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:54.293178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453658+0000","last_change":"2023-12-12T09:54:32.441630+0000","last_active":"2023-12-12T09:54:33.453658+0000","last_peered":"2023-12-12T09:54:33.453658+0000","last_clean":"2023-12-12T09:54:33.453658+0000","last_became_active":"2023-12-12T09:54:32.441384+0000","last_became_peered":"2023-12-12T09:54:32.441384+0000","last_unstale":"2023-12-12T09:54:33.453658+0000","last_undegraded":"2023-12-12T09:54:33.453658+0000","last_fullsized":"2023-12-12T09:54:33.453658+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:44:16.374018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.460216+0000","last_change":"2023-12-12T09:54:33.449161+0000","last_active":"2023-12-12T09:54:33.460216+0000","last_peered":"2023-12-12T09:54:33.460216+0000","last_clean":"2023-12-12T09:54:33.460216+0000","last_became_active":"2023-12-12T09:54:33.445046+0000","last_became_peered":"2023-12-12T09:54:33.445046+0000","last_unstale":"2023-12-12T09:54:33.460216+0000","last_undegraded":"2023-12-12T09:54:33.460216+0000","last_fullsized":"2023-12-12T09:54:33.460216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:01:17.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.426742+0000","last_change":"2023-12-12T09:54:32.441231+0000","last_active":"2023-12-12T09:54:33.426742+0000","last_peered":"2023-12-12T09:54:33.426742+0000","last_clean":"2023-12-12T09:54:33.426742+0000","last_became_active":"2023-12-12T09:54:32.440922+0000","last_became_peered":"2023-12-12T09:54:32.440922+0000","last_unstale":"2023-12-12T09:54:33.426742+0000","last_undegraded":"2023-12-12T09:54:33.426742+0000","last_fullsized":"2023-12-12T09:54:33.426742+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:31:08.564303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.455739+0000","last_change":"2023-12-12T09:54:33.454433+0000","last_active":"2023-12-12T09:54:33.455739+0000","last_peered":"2023-12-12T09:54:33.455739+0000","last_clean":"2023-12-12T09:54:33.455739+0000","last_became_active":"2023-12-12T09:54:33.454177+0000","last_became_peered":"2023-12-12T09:54:33.454177+0000","last_unstale":"2023-12-12T09:54:33.455739+0000","last_undegraded":"2023-12-12T09:54:33.455739+0000","last_fullsized":"2023-12-12T09:54:33.455739+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:41:48.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.841998+0000","last_change":"2023-12-12T09:54:32.442404+0000","last_active":"2023-12-12T09:54:34.841998+0000","last_peered":"2023-12-12T09:54:34.841998+0000","last_clean":"2023-12-12T09:54:34.841998+0000","last_became_active":"2023-12-12T09:54:32.442112+0000","last_became_peered":"2023-12-12T09:54:32.442112+0000","last_unstale":"2023-12-12T09:54:34.841998+0000","last_undegraded":"2023-12-12T09:54:34.841998+0000","last_fullsized":"2023-12-12T09:54:34.841998+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:16:24.728525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.447552+0000","last_change":"2023-12-12T09:54:33.439390+0000","last_active":"2023-12-12T09:54:33.447552+0000","last_peered":"2023-12-12T09:54:33.447552+0000","last_clean":"2023-12-12T09:54:33.447552+0000","last_became_active":"2023-12-12T09:54:33.439053+0000","last_became_peered":"2023-12-12T09:54:33.439053+0000","last_unstale":"2023-12-12T09:54:33.447552+0000","last_undegraded":"2023-12-12T09:54:33.447552+0000","last_fullsized":"2023-12-12T09:54:33.447552+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:03.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.461918+0000","last_change":"2023-12-12T09:54:32.442971+0000","last_active":"2023-12-12T09:54:33.461918+0000","last_peered":"2023-12-12T09:54:33.461918+0000","last_clean":"2023-12-12T09:54:33.461918+0000","last_became_active":"2023-12-12T09:54:32.442684+0000","last_became_peered":"2023-12-12T09:54:32.442684+0000","last_unstale":"2023-12-12T09:54:33.461918+0000","last_undegraded":"2023-12-12T09:54:33.461918+0000","last_fullsized":"2023-12-12T09:54:33.461918+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:53.278068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.457474+0000","last_change":"2023-12-12T09:54:33.455246+0000","last_active":"2023-12-12T09:54:33.457474+0000","last_peered":"2023-12-12T09:54:33.457474+0000","last_clean":"2023-12-12T09:54:33.457474+0000","last_became_active":"2023-12-12T09:54:33.454951+0000","last_became_peered":"2023-12-12T09:54:33.454951+0000","last_unstale":"2023-12-12T09:54:33.457474+0000","last_undegraded":"2023-12-12T09:54:33.457474+0000","last_fullsized":"2023-12-12T09:54:33.457474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:03.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.459467+0000","last_change":"2023-12-12T09:54:32.443691+0000","last_active":"2023-12-12T09:54:33.459467+0000","last_peered":"2023-12-12T09:54:33.459467+0000","last_clean":"2023-12-12T09:54:33.459467+0000","last_became_active":"2023-12-12T09:54:32.443409+0000","last_became_peered":"2023-12-12T09:54:32.443409+0000","last_unstale":"2023-12-12T09:54:33.459467+0000","last_undegraded":"2023-12-12T09:54:33.459467+0000","last_fullsized":"2023-12-12T09:54:33.459467+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:45:12.022632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449479+0000","last_change":"2023-12-12T09:54:33.442991+0000","last_active":"2023-12-12T09:54:33.449479+0000","last_peered":"2023-12-12T09:54:33.449479+0000","last_clean":"2023-12-12T09:54:33.449479+0000","last_became_active":"2023-12-12T09:54:33.442408+0000","last_became_peered":"2023-12-12T09:54:33.442408+0000","last_unstale":"2023-12-12T09:54:33.449479+0000","last_undegraded":"2023-12-12T09:54:33.449479+0000","last_fullsized":"2023-12-12T09:54:33.449479+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:28:27.268634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.446648+0000","last_change":"2023-12-12T09:54:32.440300+0000","last_active":"2023-12-12T09:54:34.446648+0000","last_peered":"2023-12-12T09:54:34.446648+0000","last_clean":"2023-12-12T09:54:34.446648+0000","last_became_active":"2023-12-12T09:54:32.436850+0000","last_became_peered":"2023-12-12T09:54:32.436850+0000","last_unstale":"2023-12-12T09:54:34.446648+0000","last_undegraded":"2023-12-12T09:54:34.446648+0000","last_fullsized":"2023-12-12T09:54:34.446648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.343227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.462525+0000","last_change":"2023-12-12T09:54:33.461081+0000","last_active":"2023-12-12T09:54:33.462525+0000","last_peered":"2023-12-12T09:54:33.462525+0000","last_clean":"2023-12-12T09:54:33.462525+0000","last_became_active":"2023-12-12T09:54:33.460796+0000","last_became_peered":"2023-12-12T09:54:33.460796+0000","last_unstale":"2023-12-12T09:54:33.462525+0000","last_undegraded":"2023-12-12T09:54:33.462525+0000","last_fullsized":"2023-12-12T09:54:33.462525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:30:50.396953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451488+0000","last_change":"2023-12-12T09:54:32.447889+0000","last_active":"2023-12-12T09:54:33.451488+0000","last_peered":"2023-12-12T09:54:33.451488+0000","last_clean":"2023-12-12T09:54:33.451488+0000","last_became_active":"2023-12-12T09:54:32.447665+0000","last_became_peered":"2023-12-12T09:54:32.447665+0000","last_unstale":"2023-12-12T09:54:33.451488+0000","last_undegraded":"2023-12-12T09:54:33.451488+0000","last_fullsized":"2023-12-12T09:54:33.451488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:33:55.675558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.437527+0000","last_change":"2023-12-12T09:54:33.429814+0000","last_active":"2023-12-12T09:54:33.437527+0000","last_peered":"2023-12-12T09:54:33.437527+0000","last_clean":"2023-12-12T09:54:33.437527+0000","last_became_active":"2023-12-12T09:54:33.429505+0000","last_became_peered":"2023-12-12T09:54:33.429505+0000","last_unstale":"2023-12-12T09:54:33.437527+0000","last_undegraded":"2023-12-12T09:54:33.437527+0000","last_fullsized":"2023-12-12T09:54:33.437527+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:32.422515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.425477+0000","last_change":"2023-12-12T09:54:32.447519+0000","last_active":"2023-12-12T09:54:33.425477+0000","last_peered":"2023-12-12T09:54:33.425477+0000","last_clean":"2023-12-12T09:54:33.425477+0000","last_became_active":"2023-12-12T09:54:32.447167+0000","last_became_peered":"2023-12-12T09:54:32.447167+0000","last_unstale":"2023-12-12T09:54:33.425477+0000","last_undegraded":"2023-12-12T09:54:33.425477+0000","last_fullsized":"2023-12-12T09:54:33.425477+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:20.188945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.449451+0000","last_change":"2023-12-12T09:54:33.446867+0000","last_active":"2023-12-12T09:54:34.449451+0000","last_peered":"2023-12-12T09:54:34.449451+0000","last_clean":"2023-12-12T09:54:34.449451+0000","last_became_active":"2023-12-12T09:54:33.446606+0000","last_became_peered":"2023-12-12T09:54:33.446606+0000","last_unstale":"2023-12-12T09:54:34.449451+0000","last_undegraded":"2023-12-12T09:54:34.449451+0000","last_fullsized":"2023-12-12T09:54:34.449451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:14:27.591850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.456241+0000","last_change":"2023-12-12T09:54:32.443527+0000","last_active":"2023-12-12T09:54:33.456241+0000","last_peered":"2023-12-12T09:54:33.456241+0000","last_clean":"2023-12-12T09:54:33.456241+0000","last_became_active":"2023-12-12T09:54:32.442564+0000","last_became_peered":"2023-12-12T09:54:32.442564+0000","last_unstale":"2023-12-12T09:54:33.456241+0000","last_undegraded":"2023-12-12T09:54:33.456241+0000","last_fullsized":"2023-12-12T09:54:33.456241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:31:08.564303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451814+0000","last_change":"2023-12-12T09:54:33.444637+0000","last_active":"2023-12-12T09:54:33.451814+0000","last_peered":"2023-12-12T09:54:33.451814+0000","last_clean":"2023-12-12T09:54:33.451814+0000","last_became_active":"2023-12-12T09:54:33.444346+0000","last_became_peered":"2023-12-12T09:54:33.444346+0000","last_unstale":"2023-12-12T09:54:33.451814+0000","last_undegraded":"2023-12-12T09:54:33.451814+0000","last_fullsized":"2023-12-12T09:54:33.451814+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:22:11.410125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.432357+0000","last_change":"2023-12-12T09:54:32.441702+0000","last_active":"2023-12-12T09:54:33.432357+0000","last_peered":"2023-12-12T09:54:33.432357+0000","last_clean":"2023-12-12T09:54:33.432357+0000","last_became_active":"2023-12-12T09:54:32.439279+0000","last_became_peered":"2023-12-12T09:54:32.439279+0000","last_unstale":"2023-12-12T09:54:33.432357+0000","last_undegraded":"2023-12-12T09:54:33.432357+0000","last_fullsized":"2023-12-12T09:54:33.432357+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:44:16.374018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449872+0000","last_change":"2023-12-12T09:54:33.433513+0000","last_active":"2023-12-12T09:54:33.449872+0000","last_peered":"2023-12-12T09:54:33.449872+0000","last_clean":"2023-12-12T09:54:33.449872+0000","last_became_active":"2023-12-12T09:54:33.433254+0000","last_became_peered":"2023-12-12T09:54:33.433254+0000","last_unstale":"2023-12-12T09:54:33.449872+0000","last_undegraded":"2023-12-12T09:54:33.449872+0000","last_fullsized":"2023-12-12T09:54:33.449872+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:31:09.579413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.429215+0000","last_change":"2023-12-12T09:54:32.440487+0000","last_active":"2023-12-12T09:54:33.429215+0000","last_peered":"2023-12-12T09:54:33.429215+0000","last_clean":"2023-12-12T09:54:33.429215+0000","last_became_active":"2023-12-12T09:54:32.438147+0000","last_became_peered":"2023-12-12T09:54:32.438147+0000","last_unstale":"2023-12-12T09:54:33.429215+0000","last_undegraded":"2023-12-12T09:54:33.429215+0000","last_fullsized":"2023-12-12T09:54:33.429215+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:53.278068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.456243+0000","last_change":"2023-12-12T09:54:33.451587+0000","last_active":"2023-12-12T09:54:33.456243+0000","last_peered":"2023-12-12T09:54:33.456243+0000","last_clean":"2023-12-12T09:54:33.456243+0000","last_became_active":"2023-12-12T09:54:33.450261+0000","last_became_peered":"2023-12-12T09:54:33.450261+0000","last_unstale":"2023-12-12T09:54:33.456243+0000","last_undegraded":"2023-12-12T09:54:33.456243+0000","last_fullsized":"2023-12-12T09:54:33.456243+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:18:52.128679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.445716+0000","last_change":"2023-12-12T09:54:32.441928+0000","last_active":"2023-12-12T09:54:34.445716+0000","last_peered":"2023-12-12T09:54:34.445716+0000","last_clean":"2023-12-12T09:54:34.445716+0000","last_became_active":"2023-12-12T09:54:32.441630+0000","last_became_peered":"2023-12-12T09:54:32.441630+0000","last_unstale":"2023-12-12T09:54:34.445716+0000","last_undegraded":"2023-12-12T09:54:34.445716+0000","last_fullsized":"2023-12-12T09:54:34.445716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:55:02.941021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448308+0000","last_change":"2023-12-12T09:54:33.453387+0000","last_active":"2023-12-12T09:54:34.448308+0000","last_peered":"2023-12-12T09:54:34.448308+0000","last_clean":"2023-12-12T09:54:34.448308+0000","last_became_active":"2023-12-12T09:54:33.453014+0000","last_became_peered":"2023-12-12T09:54:33.453014+0000","last_unstale":"2023-12-12T09:54:34.448308+0000","last_undegraded":"2023-12-12T09:54:34.448308+0000","last_fullsized":"2023-12-12T09:54:34.448308+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:10:40.233777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.447257+0000","last_change":"2023-12-12T09:54:32.446155+0000","last_active":"2023-12-12T09:54:33.447257+0000","last_peered":"2023-12-12T09:54:33.447257+0000","last_clean":"2023-12-12T09:54:33.447257+0000","last_became_active":"2023-12-12T09:54:32.445951+0000","last_became_peered":"2023-12-12T09:54:32.445951+0000","last_unstale":"2023-12-12T09:54:33.447257+0000","last_undegraded":"2023-12-12T09:54:33.447257+0000","last_fullsized":"2023-12-12T09:54:33.447257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:33:24.112633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449450+0000","last_change":"2023-12-12T09:54:33.437803+0000","last_active":"2023-12-12T09:54:33.449450+0000","last_peered":"2023-12-12T09:54:33.449450+0000","last_clean":"2023-12-12T09:54:33.449450+0000","last_became_active":"2023-12-12T09:54:33.436748+0000","last_became_peered":"2023-12-12T09:54:33.436748+0000","last_unstale":"2023-12-12T09:54:33.449450+0000","last_undegraded":"2023-12-12T09:54:33.449450+0000","last_fullsized":"2023-12-12T09:54:33.449450+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:21.204054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842736+0000","last_change":"2023-12-12T09:54:32.443678+0000","last_active":"2023-12-12T09:54:34.842736+0000","last_peered":"2023-12-12T09:54:34.842736+0000","last_clean":"2023-12-12T09:54:34.842736+0000","last_became_active":"2023-12-12T09:54:32.443425+0000","last_became_peered":"2023-12-12T09:54:32.443425+0000","last_unstale":"2023-12-12T09:54:34.842736+0000","last_undegraded":"2023-12-12T09:54:34.842736+0000","last_fullsized":"2023-12-12T09:54:34.842736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:00:21.640709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453793+0000","last_change":"2023-12-12T09:54:33.450758+0000","last_active":"2023-12-12T09:54:33.453793+0000","last_peered":"2023-12-12T09:54:33.453793+0000","last_clean":"2023-12-12T09:54:33.453793+0000","last_became_active":"2023-12-12T09:54:33.450255+0000","last_became_peered":"2023-12-12T09:54:33.450255+0000","last_unstale":"2023-12-12T09:54:33.453793+0000","last_undegraded":"2023-12-12T09:54:33.453793+0000","last_fullsized":"2023-12-12T09:54:33.453793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:54:26.808349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.427358+0000","last_change":"2023-12-12T09:54:32.438932+0000","last_active":"2023-12-12T09:54:33.427358+0000","last_peered":"2023-12-12T09:54:33.427358+0000","last_clean":"2023-12-12T09:54:33.427358+0000","last_became_active":"2023-12-12T09:54:32.438676+0000","last_became_peered":"2023-12-12T09:54:32.438676+0000","last_unstale":"2023-12-12T09:54:33.427358+0000","last_undegraded":"2023-12-12T09:54:33.427358+0000","last_fullsized":"2023-12-12T09:54:33.427358+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:04:16.468303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.462691+0000","last_change":"2023-12-12T09:54:33.452142+0000","last_active":"2023-12-12T09:54:33.462691+0000","last_peered":"2023-12-12T09:54:33.462691+0000","last_clean":"2023-12-12T09:54:33.462691+0000","last_became_active":"2023-12-12T09:54:33.451867+0000","last_became_peered":"2023-12-12T09:54:33.451867+0000","last_unstale":"2023-12-12T09:54:33.462691+0000","last_undegraded":"2023-12-12T09:54:33.462691+0000","last_fullsized":"2023-12-12T09:54:33.462691+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:18:15.115398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.430534+0000","last_change":"2023-12-12T09:54:32.444994+0000","last_active":"2023-12-12T09:54:33.430534+0000","last_peered":"2023-12-12T09:54:33.430534+0000","last_clean":"2023-12-12T09:54:33.430534+0000","last_became_active":"2023-12-12T09:54:32.444575+0000","last_became_peered":"2023-12-12T09:54:32.444575+0000","last_unstale":"2023-12-12T09:54:33.430534+0000","last_undegraded":"2023-12-12T09:54:33.430534+0000","last_fullsized":"2023-12-12T09:54:33.430534+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:10:39.218668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448752+0000","last_change":"2023-12-12T09:54:33.447622+0000","last_active":"2023-12-12T09:54:34.448752+0000","last_peered":"2023-12-12T09:54:34.448752+0000","last_clean":"2023-12-12T09:54:34.448752+0000","last_became_active":"2023-12-12T09:54:33.446865+0000","last_became_peered":"2023-12-12T09:54:33.446865+0000","last_unstale":"2023-12-12T09:54:34.448752+0000","last_undegraded":"2023-12-12T09:54:34.448752+0000","last_fullsized":"2023-12-12T09:54:34.448752+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:41:48.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.441271+0000","last_change":"2023-12-12T09:54:32.446927+0000","last_active":"2023-12-12T09:54:33.441271+0000","last_peered":"2023-12-12T09:54:33.441271+0000","last_clean":"2023-12-12T09:54:33.441271+0000","last_became_active":"2023-12-12T09:54:32.446519+0000","last_became_peered":"2023-12-12T09:54:32.446519+0000","last_unstale":"2023-12-12T09:54:33.441271+0000","last_undegraded":"2023-12-12T09:54:33.441271+0000","last_fullsized":"2023-12-12T09:54:33.441271+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:50:44.580528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449890+0000","last_change":"2023-12-12T09:54:33.436366+0000","last_active":"2023-12-12T09:54:33.449890+0000","last_peered":"2023-12-12T09:54:33.449890+0000","last_clean":"2023-12-12T09:54:33.449890+0000","last_became_active":"2023-12-12T09:54:33.436078+0000","last_became_peered":"2023-12-12T09:54:33.436078+0000","last_unstale":"2023-12-12T09:54:33.449890+0000","last_undegraded":"2023-12-12T09:54:33.449890+0000","last_fullsized":"2023-12-12T09:54:33.449890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:16:25.743635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.440140+0000","last_change":"2023-12-12T09:54:32.449896+0000","last_active":"2023-12-12T09:54:33.440140+0000","last_peered":"2023-12-12T09:54:33.440140+0000","last_clean":"2023-12-12T09:54:33.440140+0000","last_became_active":"2023-12-12T09:54:32.449681+0000","last_became_peered":"2023-12-12T09:54:32.449681+0000","last_unstale":"2023-12-12T09:54:33.440140+0000","last_undegraded":"2023-12-12T09:54:33.440140+0000","last_fullsized":"2023-12-12T09:54:33.440140+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:40:28.622399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.462883+0000","last_change":"2023-12-12T09:54:33.460842+0000","last_active":"2023-12-12T09:54:33.462883+0000","last_peered":"2023-12-12T09:54:33.462883+0000","last_clean":"2023-12-12T09:54:33.462883+0000","last_became_active":"2023-12-12T09:54:33.460493+0000","last_became_peered":"2023-12-12T09:54:33.460493+0000","last_unstale":"2023-12-12T09:54:33.462883+0000","last_undegraded":"2023-12-12T09:54:33.462883+0000","last_fullsized":"2023-12-12T09:54:33.462883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:18:12.264577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448527+0000","last_change":"2023-12-12T09:54:32.433995+0000","last_active":"2023-12-12T09:54:34.448527+0000","last_peered":"2023-12-12T09:54:34.448527+0000","last_clean":"2023-12-12T09:54:34.448527+0000","last_became_active":"2023-12-12T09:54:32.433337+0000","last_became_peered":"2023-12-12T09:54:32.433337+0000","last_unstale":"2023-12-12T09:54:34.448527+0000","last_undegraded":"2023-12-12T09:54:34.448527+0000","last_fullsized":"2023-12-12T09:54:34.448527+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:23.142349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.437597+0000","last_change":"2023-12-12T09:54:33.430515+0000","last_active":"2023-12-12T09:54:33.437597+0000","last_peered":"2023-12-12T09:54:33.437597+0000","last_clean":"2023-12-12T09:54:33.437597+0000","last_became_active":"2023-12-12T09:54:33.429801+0000","last_became_peered":"2023-12-12T09:54:33.429801+0000","last_unstale":"2023-12-12T09:54:33.437597+0000","last_undegraded":"2023-12-12T09:54:33.437597+0000","last_fullsized":"2023-12-12T09:54:33.437597+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:28:27.268634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.425187+0000","last_change":"2023-12-12T09:54:32.440055+0000","last_active":"2023-12-12T09:54:33.425187+0000","last_peered":"2023-12-12T09:54:33.425187+0000","last_clean":"2023-12-12T09:54:33.425187+0000","last_became_active":"2023-12-12T09:54:32.439758+0000","last_became_peered":"2023-12-12T09:54:32.439758+0000","last_unstale":"2023-12-12T09:54:33.425187+0000","last_undegraded":"2023-12-12T09:54:33.425187+0000","last_fullsized":"2023-12-12T09:54:33.425187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:00:21.640709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.441517+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:54:33.441517+0000","last_peered":"2023-12-12T09:54:33.441517+0000","last_clean":"2023-12-12T09:54:33.441517+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:54:33.441517+0000","last_undegraded":"2023-12-12T09:54:33.441517+0000","last_fullsized":"2023-12-12T09:54:33.441517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":858340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0840000000000001,"15min":1.0880000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.131,"15min":1.1339999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.7110000000000001,"5min":1.855,"15min":1.855},"last":1.5349999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0189999999999999,"15min":1.018},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.4990000000000001,"5min":1.571,"15min":1.571},"last":1.036},{"interface":"front","average":{"1min":1.139,"5min":1.1439999999999999,"15min":1.145},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5720000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.96199999999999997}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.367,"15min":1.395},"min":{"1min":0.82099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.78,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.206},{"interface":"front","average":{"1min":1.153,"5min":1.421,"15min":1.4650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6379999999999999,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.2729999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.351,"15min":1.361},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.899,"5min":2.23,"15min":2.23},"last":1.581},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.397,"15min":1.4139999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9850000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.845}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.411,"15min":1.4359999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.0449999999999999,"5min":2.649,"15min":2.649},"last":1.76},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.4079999999999999,"15min":1.4279999999999999},"min":{"1min":0.71399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7649999999999999,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.4419999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.54,"15min":1.5700000000000001},"min":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.784,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.381},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5529999999999999,"15min":1.5649999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.141,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.901}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.605,"15min":1.623},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0779999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.494},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.599,"15min":1.627},"min":{"1min":0.82599999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9370000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.6299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":602112,"data_stored":261880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.054,"15min":1.046},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.202},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1060000000000001,"15min":1.1040000000000001},"min":{"1min":0.76200000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.6200000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.77100000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6579999999999999,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.887,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2749999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1399999999999999,"15min":1.1339999999999999},"min":{"1min":0.82799999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.145},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.1140000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.3280000000000001,"15min":1.3140000000000001},"min":{"1min":1.0389999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.738},{"interface":"front","average":{"1min":1.304,"5min":1.264,"15min":1.2569999999999999},"min":{"1min":0.88600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.776,"5min":1.994,"15min":1.994},"last":1.339}]},{"osd":4,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.76500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.827,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.26},{"interface":"front","average":{"1min":1.496,"5min":1.536,"15min":1.5429999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2890000000000001,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.829}]},{"osd":5,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3859999999999999,"15min":1.3779999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.98,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.45},{"interface":"front","average":{"1min":1.55,"5min":1.595,"15min":1.603},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":2.1459999999999999,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.377}]},{"osd":7,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":1.7030000000000001,"15min":1.7090000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.4119999999999999,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.885},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.52,"15min":1.5129999999999999},"min":{"1min":0.98599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.246,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.5129999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":606208,"data_stored":267848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0429999999999999,"15min":1.026},"min":{"1min":0.71799999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0750000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.228,"15min":1.2509999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5249999999999999,"5min":3.694,"15min":3.694},"last":0.78600000000000003}]},{"osd":1,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.3220000000000001,"15min":1.3480000000000001},"min":{"1min":0.76300000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7470000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.135},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0880000000000001,"15min":1.0960000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.0900000000000001,"15min":1.0760000000000001},"min":{"1min":0.91400000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.115,"15min":1.0940000000000001},"min":{"1min":0.874,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.202}]},{"osd":3,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.361,"15min":1.371},"min":{"1min":0.79100000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.74,"5min":3.944,"15min":3.944},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.298,"15min":1.2929999999999999},"min":{"1min":0.92500000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.8260000000000001,"5min":1.925,"15min":1.925},"last":1.369}]},{"osd":4,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.341,"15min":1.335},"min":{"1min":1.0209999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0150000000000001,"5min":2.536,"15min":2.536},"last":1.48},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.53,"15min":1.554},"min":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8420000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.4370000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.462,"15min":1.464},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9430000000000001,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.518},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5009999999999999,"15min":1.4990000000000001},"min":{"1min":1.0649999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.8839999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.621,"15min":1.633},"min":{"1min":1.095,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9870000000000001,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.647,"15min":1.6639999999999999},"min":{"1min":1.1040000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.048,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.6040000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":593920,"data_stored":260791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.095,"15min":1.077},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":5.5430000000000001,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":1.028},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.196,"15min":1.1160000000000001},"min":{"1min":0.66500000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":7.29,"5min":7.29,"15min":7.29},"last":1.472}]},{"osd":1,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.2290000000000001,"15min":1.171},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.3360000000000003,"5min":7.3360000000000003,"15min":7.3360000000000003},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2949999999999999,"15min":1.288},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.7530000000000001,"5min":6.7530000000000001,"15min":6.7530000000000001},"last":1.5009999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.165,"15min":1.1539999999999999},"min":{"1min":0.67700000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.891,"5min":4.891,"15min":4.891},"last":1.341},{"interface":"front","average":{"1min":1.742,"5min":1.373,"15min":1.3109999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":7.4100000000000001,"5min":7.4100000000000001,"15min":7.4100000000000001},"last":1.3600000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3200000000000001,"15min":1.306},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.337,"15min":1.347},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.7409999999999997,"5min":4.7409999999999997,"15min":4.7409999999999997},"last":1.264}]},{"osd":5,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.9379999999999999,"5min":1.502,"15min":1.429},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.508,"15min":1.49},"min":{"1min":0.872,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":6.8499999999999996,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":1.2889999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.409,"15min":1.405},"min":{"1min":0.64800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.619,"15min":1.5620000000000001},"min":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":7.3739999999999997,"5min":7.3739999999999997,"15min":7.3739999999999997},"last":0.95299999999999996}]},{"osd":7,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.7030000000000001,"5min":1.5880000000000001,"15min":1.569},"min":{"1min":0.93799999999999994,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.823,"5min":1.613,"15min":1.5780000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":6.4260000000000002,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.137}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":610304,"data_stored":262862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81200000000000006,"15min":0.80600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.879,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.484,"5min":1.577,"15min":1.577},"last":0.99399999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.88500000000000001,"15min":0.875},"min":{"1min":0.54600000000000004,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4630000000000001,"5min":1.548,"15min":1.548},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96999999999999997,"15min":0.97399999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4279999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.55}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.05,"15min":1.0640000000000001},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.6060000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.6459999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.645,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.679},{"interface":"front","average":{"1min":1.196,"5min":1.252,"15min":1.262},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6870000000000001,"5min":1.964,"15min":1.964},"last":1.5980000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2949999999999999,"15min":1.3149999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.669,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3160000000000001,"15min":1.319},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.885,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.748}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3280000000000001,"15min":1.3320000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.738,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.228},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3169999999999999,"15min":1.327},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6499999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4330000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3440000000000001,"15min":1.351},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.7949999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.387},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.4730000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":618496,"data_stored":269460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.83599999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.89400000000000002,"15min":0.91100000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2230000000000001,"5min":1.752,"15min":1.752},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.90700000000000003,"15min":0.92300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4550000000000001,"5min":1.603,"15min":1.603},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.95499999999999996,"15min":0.96999999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4019999999999999,"5min":1.633,"15min":1.633},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.89900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.1559999999999999,"15min":1.1839999999999999},"min":{"1min":0.53800000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8879999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.63200000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.2989999999999999,"15min":1.3220000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.254,"15min":1.26},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.629,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.872}]},{"osd":5,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.1870000000000001,"15min":1.2110000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.506,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3220000000000001},"min":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.645,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.92200000000000004}]},{"osd":6,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.369,"15min":1.3919999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.631,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.2010000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.038}]},{"osd":7,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.452,"15min":1.48},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6779999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.399,"15min":1.429},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1139999999999999,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.0660000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":260369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.153,"5min":1.486,"15min":1.486},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90800000000000003,"15min":0.91900000000000004},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3520000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.78300000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.879,"15min":0.875},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.579,"5min":1.712,"15min":1.712},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.99199999999999999,"15min":0.99299999999999999},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.232}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.018,"15min":1.036},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3919999999999999,"5min":1.665,"15min":1.665},"last":1.333},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":0.98999999999999999},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4570000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.123,"15min":1.1319999999999999},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4570000000000001,"5min":2.056,"15min":2.056},"last":1.262},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2929999999999999,"15min":1.3089999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":2.427,"15min":2.427},"last":1.399}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.363,"15min":1.365},"min":{"1min":1.05,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8129999999999999,"5min":3.476,"15min":3.476},"last":1.472},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.3300000000000001,"15min":1.3500000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.7010000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.5109999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4139999999999999,"15min":1.4279999999999999},"min":{"1min":0.94699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7230000000000001,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.625},{"interface":"front","average":{"1min":1.198,"5min":1.3380000000000001,"15min":1.3620000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.573,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.5960000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3839999999999999,"15min":1.3979999999999999},"min":{"1min":0.83199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.694,"5min":3.052,"15min":3.052},"last":1.556},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4379999999999999,"15min":1.4510000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9079999999999999,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":1.3700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":850633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3999999999999999,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3420000000000001,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.56999999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.165,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0600000000000001,"15min":1.0860000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.5089999999999999,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.98199999999999998},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":1.296,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.661,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.57699999999999996}]},{"osd":4,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1779999999999999,"15min":1.1890000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.577,"5min":2.738,"15min":2.738},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.171,"5min":1.3300000000000001,"15min":1.3560000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.528,"5min":3.875,"15min":3.875},"last":1.016}]},{"osd":5,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2589999999999999,"15min":1.2729999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.625,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2749999999999999,"15min":1.2909999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5680000000000001,"5min":2.54,"15min":2.54},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.698,"5min":2.117,"15min":2.117},"last":1.002},{"interface":"front","average":{"1min":1.282,"5min":1.4179999999999999,"15min":1.4399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7230000000000001,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.80700000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3779999999999999,"15min":1.3859999999999999},"min":{"1min":0.84699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7629999999999999,"5min":2.931,"15min":2.931},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6399999999999999,"5min":4.548,"15min":4.548},"last":0.92000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:54:37.894 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:54:38.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.151+0000 7fe54511d640 1 Processor -- start 2023-12-12T09:54:38.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.151+0000 7fe54511d640 1 -- start start 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.152+0000 7fe54511d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540143e00 0x7fe5400b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.152+0000 7fe54511d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe540072180 0x7fe5400b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe54511d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe54006c410 0x7fe5400b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe54511d640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5400b3bb0 con 0x7fe54006c410 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe54511d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5401441e0 con 0x7fe540072180 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe54511d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe540072680 con 0x7fe540143e00 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53f59e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe54006c410 0x7fe5400b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53f59e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe54006c410 0x7fe5400b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49872/0 (socket says 172.21.15.72:49872) 2023-12-12T09:54:38.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53f59e640 1 -- 172.21.15.72:0/2870815734 learned_addr learned my addr 172.21.15.72:0/2870815734 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:38.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53ed9d640 1 --2- 172.21.15.72:0/2870815734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540143e00 0x7fe5400b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53e59c640 1 --2- 172.21.15.72:0/2870815734 >> v2:172.21.15.113:3300/0 conn(0x7fe540072180 0x7fe5400b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53f59e640 1 -- 172.21.15.72:0/2870815734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540143e00 msgr2=0x7fe5400b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:38.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53f59e640 1 --2- 172.21.15.72:0/2870815734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540143e00 0x7fe5400b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53f59e640 1 -- 172.21.15.72:0/2870815734 >> v2:172.21.15.113:3300/0 conn(0x7fe540072180 msgr2=0x7fe5400b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:38.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53f59e640 1 --2- 172.21.15.72:0/2870815734 >> v2:172.21.15.113:3300/0 conn(0x7fe540072180 0x7fe5400b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.153+0000 7fe53f59e640 1 -- 172.21.15.72:0/2870815734 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5400b1e30 con 0x7fe54006c410 2023-12-12T09:54:38.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.155+0000 7fe53f59e640 1 --2- 172.21.15.72:0/2870815734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400b4ac0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe54006b810 tx=0x7fe53802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c3fd890966bb68 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.155+0000 7fe53dd9b640 1 -- 172.21.15.72:0/2870815734 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53802ca70 con 0x7fe54006c410 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.155+0000 7fe53dd9b640 1 -- 172.21.15.72:0/2870815734 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53802cbd0 con 0x7fe54006c410 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.155+0000 7fe53dd9b640 1 -- 172.21.15.72:0/2870815734 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380343f0 con 0x7fe54006c410 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.155+0000 7fe54511d640 1 -- 172.21.15.72:0/2870815734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 msgr2=0x7fe5400b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.155+0000 7fe54511d640 1 --2- 172.21.15.72:0/2870815734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400b4ac0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe54006b810 tx=0x7fe53802bed0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.156+0000 7fe54511d640 1 -- 172.21.15.72:0/2870815734 shutdown_connections 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.156+0000 7fe54511d640 1 --2- 172.21.15.72:0/2870815734 >> v2:172.21.15.113:3300/0 conn(0x7fe540072180 0x7fe5400b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.156+0000 7fe54511d640 1 --2- 172.21.15.72:0/2870815734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400b4ac0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.156+0000 7fe54511d640 1 --2- 172.21.15.72:0/2870815734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540143e00 0x7fe5400b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.156+0000 7fe54511d640 1 -- 172.21.15.72:0/2870815734 >> 172.21.15.72:0/2870815734 conn(0x7fe5400b0a90 msgr2=0x7fe5400b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.156+0000 7fe54511d640 1 -- 172.21.15.72:0/2870815734 shutdown_connections 2023-12-12T09:54:38.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.156+0000 7fe54511d640 1 -- 172.21.15.72:0/2870815734 wait complete. 2023-12-12T09:54:38.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.157+0000 7fe54511d640 1 Processor -- start 2023-12-12T09:54:38.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.158+0000 7fe54511d640 1 -- start start 2023-12-12T09:54:38.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.158+0000 7fe54511d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.158+0000 7fe54511d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540072180 0x7fe54015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe54511d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe540143e00 0x7fe54015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe54511d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400a8e60 con 0x7fe54006c410 2023-12-12T09:54:38.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe54511d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400aa690 con 0x7fe540143e00 2023-12-12T09:54:38.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe54511d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5400a90b0 con 0x7fe540072180 2023-12-12T09:54:38.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe53ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe53ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400aa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49874/0 (socket says 172.21.15.72:49874) 2023-12-12T09:54:38.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe53ed9d640 1 -- 172.21.15.72:0/2877589996 learned_addr learned my addr 172.21.15.72:0/2877589996 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe53ed9d640 1 -- 172.21.15.72:0/2877589996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540072180 msgr2=0x7fe54015a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe53ed9d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540072180 0x7fe54015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe53ed9d640 1 -- 172.21.15.72:0/2877589996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe540143e00 msgr2=0x7fe54015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe53ed9d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe540143e00 0x7fe54015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.159+0000 7fe53ed9d640 1 -- 172.21.15.72:0/2877589996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe538009b30 con 0x7fe54006c410 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.160+0000 7fe53ed9d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400aa150 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe53000d5f0 tx=0x7fe53000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.160+0000 7fe5377fe640 1 -- 172.21.15.72:0/2877589996 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300125f0 con 0x7fe54006c410 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.160+0000 7fe5377fe640 1 -- 172.21.15.72:0/2877589996 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe530012750 con 0x7fe54006c410 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.160+0000 7fe5377fe640 1 -- 172.21.15.72:0/2877589996 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530012b00 con 0x7fe54006c410 2023-12-12T09:54:38.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.160+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5400a9390 con 0x7fe54006c410 2023-12-12T09:54:38.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.160+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5400a9950 con 0x7fe54006c410 2023-12-12T09:54:38.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.161+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5400745d0 con 0x7fe54006c410 2023-12-12T09:54:38.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.162+0000 7fe5377fe640 1 -- 172.21.15.72:0/2877589996 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe530012c60 con 0x7fe54006c410 2023-12-12T09:54:38.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.163+0000 7fe5377fe640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe52006ae40 0x7fe52006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.163+0000 7fe5377fe640 1 -- 172.21.15.72:0/2877589996 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe530089c40 con 0x7fe54006c410 2023-12-12T09:54:38.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.164+0000 7fe53e59c640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe52006ae40 0x7fe52006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.164+0000 7fe53e59c640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe52006ae40 0x7fe52006d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe528006b10 tx=0x7fe528008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:38.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.168+0000 7fe5377fe640 1 -- 172.21.15.72:0/2877589996 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe530054bb0 con 0x7fe54006c410 2023-12-12T09:54:38.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.354+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5400b23f0 con 0x7fe52006ae40 2023-12-12T09:54:38.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.373+0000 7fe5377fe640 1 -- 172.21.15.72:0/2877589996 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286962 (secure 0 0 0) 0x7fe52800ae50 con 0x7fe52006ae40 2023-12-12T09:54:38.377 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:38.382 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.382+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe52006ae40 msgr2=0x7fe52006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.382+0000 7fe54511d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe52006ae40 0x7fe52006d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe528006b10 tx=0x7fe528008040 comp rx=0 tx=0).stop 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.382+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 msgr2=0x7fe5400aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.382+0000 7fe54511d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400aa150 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe53000d5f0 tx=0x7fe53000dab0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.383+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 shutdown_connections 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.383+0000 7fe54511d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe52006ae40 0x7fe52006d300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.383+0000 7fe54511d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe540143e00 0x7fe54015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.383+0000 7fe54511d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe540072180 0x7fe54015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.383+0000 7fe54511d640 1 --2- 172.21.15.72:0/2877589996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe54006c410 0x7fe5400aa150 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.383+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 >> 172.21.15.72:0/2877589996 conn(0x7fe5400b0a90 msgr2=0x7fe540144320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.383+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 shutdown_connections 2023-12-12T09:54:38.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.383+0000 7fe54511d640 1 -- 172.21.15.72:0/2877589996 wait complete. 2023-12-12T09:54:38.405 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-12-12T09:54:37.593148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255364,"kb_used_data":5888,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719356,"statfs":{"total":773094113280,"available":772832620544,"internally_reserved":0,"allocated":6029312,"data_stored":3292183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001616"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.446501+0000","last_change":"2023-12-12T09:54:33.435866+0000","last_active":"2023-12-12T09:54:33.446501+0000","last_peered":"2023-12-12T09:54:33.446501+0000","last_clean":"2023-12-12T09:54:33.446501+0000","last_became_active":"2023-12-12T09:54:33.435542+0000","last_became_peered":"2023-12-12T09:54:33.435542+0000","last_unstale":"2023-12-12T09:54:33.446501+0000","last_undegraded":"2023-12-12T09:54:33.446501+0000","last_fullsized":"2023-12-12T09:54:33.446501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:56.972947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842499+0000","last_change":"2023-12-12T09:54:33.454431+0000","last_active":"2023-12-12T09:54:34.842499+0000","last_peered":"2023-12-12T09:54:34.842499+0000","last_clean":"2023-12-12T09:54:34.842499+0000","last_became_active":"2023-12-12T09:54:33.454105+0000","last_became_peered":"2023-12-12T09:54:33.454105+0000","last_unstale":"2023-12-12T09:54:34.842499+0000","last_undegraded":"2023-12-12T09:54:34.842499+0000","last_fullsized":"2023-12-12T09:54:34.842499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:41:48.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.458261+0000","last_change":"2023-12-12T09:54:33.441791+0000","last_active":"2023-12-12T09:54:33.458261+0000","last_peered":"2023-12-12T09:54:33.458261+0000","last_clean":"2023-12-12T09:54:33.458261+0000","last_became_active":"2023-12-12T09:54:33.441528+0000","last_became_peered":"2023-12-12T09:54:33.441528+0000","last_unstale":"2023-12-12T09:54:33.458261+0000","last_undegraded":"2023-12-12T09:54:33.458261+0000","last_fullsized":"2023-12-12T09:54:33.458261+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:24.157458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842541+0000","last_change":"2023-12-12T09:54:33.456039+0000","last_active":"2023-12-12T09:54:34.842541+0000","last_peered":"2023-12-12T09:54:34.842541+0000","last_clean":"2023-12-12T09:54:34.842541+0000","last_became_active":"2023-12-12T09:54:33.455729+0000","last_became_peered":"2023-12-12T09:54:33.455729+0000","last_unstale":"2023-12-12T09:54:34.842541+0000","last_undegraded":"2023-12-12T09:54:34.842541+0000","last_fullsized":"2023-12-12T09:54:34.842541+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:14:27.591850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.454004+0000","last_change":"2023-12-12T09:54:33.446247+0000","last_active":"2023-12-12T09:54:33.454004+0000","last_peered":"2023-12-12T09:54:33.454004+0000","last_clean":"2023-12-12T09:54:33.454004+0000","last_became_active":"2023-12-12T09:54:33.445978+0000","last_became_peered":"2023-12-12T09:54:33.445978+0000","last_unstale":"2023-12-12T09:54:33.454004+0000","last_undegraded":"2023-12-12T09:54:33.454004+0000","last_fullsized":"2023-12-12T09:54:33.454004+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:47.433030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449496+0000","last_change":"2023-12-12T09:54:33.444230+0000","last_active":"2023-12-12T09:54:33.449496+0000","last_peered":"2023-12-12T09:54:33.449496+0000","last_clean":"2023-12-12T09:54:33.449496+0000","last_became_active":"2023-12-12T09:54:33.443949+0000","last_became_peered":"2023-12-12T09:54:33.443949+0000","last_unstale":"2023-12-12T09:54:33.449496+0000","last_undegraded":"2023-12-12T09:54:33.449496+0000","last_fullsized":"2023-12-12T09:54:33.449496+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:47.433030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463641+0000","last_change":"2023-12-12T09:54:33.453152+0000","last_active":"2023-12-12T09:54:33.463641+0000","last_peered":"2023-12-12T09:54:33.463641+0000","last_clean":"2023-12-12T09:54:33.463641+0000","last_became_active":"2023-12-12T09:54:33.447789+0000","last_became_peered":"2023-12-12T09:54:33.447789+0000","last_unstale":"2023-12-12T09:54:33.463641+0000","last_undegraded":"2023-12-12T09:54:33.463641+0000","last_fullsized":"2023-12-12T09:54:33.463641+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:22:11.410125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.465627+0000","last_change":"2023-12-12T09:54:33.463606+0000","last_active":"2023-12-12T09:54:33.465627+0000","last_peered":"2023-12-12T09:54:33.465627+0000","last_clean":"2023-12-12T09:54:33.465627+0000","last_became_active":"2023-12-12T09:54:33.463339+0000","last_became_peered":"2023-12-12T09:54:33.463339+0000","last_unstale":"2023-12-12T09:54:33.465627+0000","last_undegraded":"2023-12-12T09:54:33.465627+0000","last_fullsized":"2023-12-12T09:54:33.465627+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:45:13.037742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.465638+0000","last_change":"2023-12-12T09:54:33.463635+0000","last_active":"2023-12-12T09:54:33.465638+0000","last_peered":"2023-12-12T09:54:33.465638+0000","last_clean":"2023-12-12T09:54:33.465638+0000","last_became_active":"2023-12-12T09:54:33.463399+0000","last_became_peered":"2023-12-12T09:54:33.463399+0000","last_unstale":"2023-12-12T09:54:33.465638+0000","last_undegraded":"2023-12-12T09:54:33.465638+0000","last_fullsized":"2023-12-12T09:54:33.465638+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:05:19.068458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.459062+0000","last_change":"2023-12-12T09:54:33.455702+0000","last_active":"2023-12-12T09:54:33.459062+0000","last_peered":"2023-12-12T09:54:33.459062+0000","last_clean":"2023-12-12T09:54:33.459062+0000","last_became_active":"2023-12-12T09:54:33.453529+0000","last_became_peered":"2023-12-12T09:54:33.453529+0000","last_unstale":"2023-12-12T09:54:33.459062+0000","last_undegraded":"2023-12-12T09:54:33.459062+0000","last_fullsized":"2023-12-12T09:54:33.459062+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:01:17.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.446639+0000","last_change":"2023-12-12T09:54:33.443656+0000","last_active":"2023-12-12T09:54:34.446639+0000","last_peered":"2023-12-12T09:54:34.446639+0000","last_clean":"2023-12-12T09:54:34.446639+0000","last_became_active":"2023-12-12T09:54:33.443318+0000","last_became_peered":"2023-12-12T09:54:33.443318+0000","last_unstale":"2023-12-12T09:54:34.446639+0000","last_undegraded":"2023-12-12T09:54:34.446639+0000","last_fullsized":"2023-12-12T09:54:34.446639+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:07:19.412716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451117+0000","last_change":"2023-12-12T09:54:33.444827+0000","last_active":"2023-12-12T09:54:33.451117+0000","last_peered":"2023-12-12T09:54:33.451117+0000","last_clean":"2023-12-12T09:54:33.451117+0000","last_became_active":"2023-12-12T09:54:33.444497+0000","last_became_peered":"2023-12-12T09:54:33.444497+0000","last_unstale":"2023-12-12T09:54:33.451117+0000","last_undegraded":"2023-12-12T09:54:33.451117+0000","last_fullsized":"2023-12-12T09:54:33.451117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:27:59.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.459070+0000","last_change":"2023-12-12T09:54:33.456728+0000","last_active":"2023-12-12T09:54:33.459070+0000","last_peered":"2023-12-12T09:54:33.459070+0000","last_clean":"2023-12-12T09:54:33.459070+0000","last_became_active":"2023-12-12T09:54:33.456383+0000","last_became_peered":"2023-12-12T09:54:33.456383+0000","last_unstale":"2023-12-12T09:54:33.459070+0000","last_undegraded":"2023-12-12T09:54:33.459070+0000","last_fullsized":"2023-12-12T09:54:33.459070+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:22:11.410125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453775+0000","last_change":"2023-12-12T09:54:33.442998+0000","last_active":"2023-12-12T09:54:33.453775+0000","last_peered":"2023-12-12T09:54:33.453775+0000","last_clean":"2023-12-12T09:54:33.453775+0000","last_became_active":"2023-12-12T09:54:33.442319+0000","last_became_peered":"2023-12-12T09:54:33.442319+0000","last_unstale":"2023-12-12T09:54:33.453775+0000","last_undegraded":"2023-12-12T09:54:33.453775+0000","last_fullsized":"2023-12-12T09:54:33.453775+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:12:49.054669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.841968+0000","last_change":"2023-12-12T09:54:33.449273+0000","last_active":"2023-12-12T09:54:34.841968+0000","last_peered":"2023-12-12T09:54:34.841968+0000","last_clean":"2023-12-12T09:54:34.841968+0000","last_became_active":"2023-12-12T09:54:33.448949+0000","last_became_peered":"2023-12-12T09:54:33.448949+0000","last_unstale":"2023-12-12T09:54:34.841968+0000","last_undegraded":"2023-12-12T09:54:34.841968+0000","last_fullsized":"2023-12-12T09:54:34.841968+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:03.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449031+0000","last_change":"2023-12-12T09:54:33.444507+0000","last_active":"2023-12-12T09:54:33.449031+0000","last_peered":"2023-12-12T09:54:33.449031+0000","last_clean":"2023-12-12T09:54:33.449031+0000","last_became_active":"2023-12-12T09:54:33.444256+0000","last_became_peered":"2023-12-12T09:54:33.444256+0000","last_unstale":"2023-12-12T09:54:33.449031+0000","last_undegraded":"2023-12-12T09:54:33.449031+0000","last_fullsized":"2023-12-12T09:54:33.449031+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:30:50.396953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463749+0000","last_change":"2023-12-12T09:54:33.459282+0000","last_active":"2023-12-12T09:54:33.463749+0000","last_peered":"2023-12-12T09:54:33.463749+0000","last_clean":"2023-12-12T09:54:33.463749+0000","last_became_active":"2023-12-12T09:54:33.449272+0000","last_became_peered":"2023-12-12T09:54:33.449272+0000","last_unstale":"2023-12-12T09:54:33.463749+0000","last_undegraded":"2023-12-12T09:54:33.463749+0000","last_fullsized":"2023-12-12T09:54:33.463749+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:10:40.233777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842683+0000","last_change":"2023-12-12T09:54:33.454480+0000","last_active":"2023-12-12T09:54:34.842683+0000","last_peered":"2023-12-12T09:54:34.842683+0000","last_clean":"2023-12-12T09:54:34.842683+0000","last_became_active":"2023-12-12T09:54:33.454194+0000","last_became_peered":"2023-12-12T09:54:33.454194+0000","last_unstale":"2023-12-12T09:54:34.842683+0000","last_undegraded":"2023-12-12T09:54:34.842683+0000","last_fullsized":"2023-12-12T09:54:34.842683+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:44:17.389128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463632+0000","last_change":"2023-12-12T09:54:33.461140+0000","last_active":"2023-12-12T09:54:33.463632+0000","last_peered":"2023-12-12T09:54:33.463632+0000","last_clean":"2023-12-12T09:54:33.463632+0000","last_became_active":"2023-12-12T09:54:33.460869+0000","last_became_peered":"2023-12-12T09:54:33.460869+0000","last_unstale":"2023-12-12T09:54:33.463632+0000","last_undegraded":"2023-12-12T09:54:33.463632+0000","last_fullsized":"2023-12-12T09:54:33.463632+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:27:59.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451947+0000","last_change":"2023-12-12T09:54:33.444494+0000","last_active":"2023-12-12T09:54:33.451947+0000","last_peered":"2023-12-12T09:54:33.451947+0000","last_clean":"2023-12-12T09:54:33.451947+0000","last_became_active":"2023-12-12T09:54:33.444120+0000","last_became_peered":"2023-12-12T09:54:33.444120+0000","last_unstale":"2023-12-12T09:54:33.451947+0000","last_undegraded":"2023-12-12T09:54:33.451947+0000","last_fullsized":"2023-12-12T09:54:33.451947+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:18:52.128679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.445509+0000","last_change":"2023-12-12T09:54:33.442029+0000","last_active":"2023-12-12T09:54:33.445509+0000","last_peered":"2023-12-12T09:54:33.445509+0000","last_clean":"2023-12-12T09:54:33.445509+0000","last_became_active":"2023-12-12T09:54:33.441737+0000","last_became_peered":"2023-12-12T09:54:33.441737+0000","last_unstale":"2023-12-12T09:54:33.445509+0000","last_undegraded":"2023-12-12T09:54:33.445509+0000","last_fullsized":"2023-12-12T09:54:33.445509+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:01:17.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463428+0000","last_change":"2023-12-12T09:54:33.462336+0000","last_active":"2023-12-12T09:54:33.463428+0000","last_peered":"2023-12-12T09:54:33.463428+0000","last_clean":"2023-12-12T09:54:33.463428+0000","last_became_active":"2023-12-12T09:54:33.462056+0000","last_became_peered":"2023-12-12T09:54:33.462056+0000","last_unstale":"2023-12-12T09:54:33.463428+0000","last_undegraded":"2023-12-12T09:54:33.463428+0000","last_fullsized":"2023-12-12T09:54:33.463428+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:18:15.115398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.445833+0000","last_change":"2023-12-12T09:54:33.440468+0000","last_active":"2023-12-12T09:54:34.445833+0000","last_peered":"2023-12-12T09:54:34.445833+0000","last_clean":"2023-12-12T09:54:34.445833+0000","last_became_active":"2023-12-12T09:54:33.440190+0000","last_became_peered":"2023-12-12T09:54:33.440190+0000","last_unstale":"2023-12-12T09:54:34.445833+0000","last_undegraded":"2023-12-12T09:54:34.445833+0000","last_fullsized":"2023-12-12T09:54:34.445833+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:55:48.144356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463027+0000","last_change":"2023-12-12T09:54:33.452024+0000","last_active":"2023-12-12T09:54:33.463027+0000","last_peered":"2023-12-12T09:54:33.463027+0000","last_clean":"2023-12-12T09:54:33.463027+0000","last_became_active":"2023-12-12T09:54:33.451593+0000","last_became_peered":"2023-12-12T09:54:33.451593+0000","last_unstale":"2023-12-12T09:54:33.463027+0000","last_undegraded":"2023-12-12T09:54:33.463027+0000","last_fullsized":"2023-12-12T09:54:33.463027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:17:41.494785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.463768+0000","last_change":"2023-12-12T09:54:33.459917+0000","last_active":"2023-12-12T09:54:33.463768+0000","last_peered":"2023-12-12T09:54:33.463768+0000","last_clean":"2023-12-12T09:54:33.463768+0000","last_became_active":"2023-12-12T09:54:33.459595+0000","last_became_peered":"2023-12-12T09:54:33.459595+0000","last_unstale":"2023-12-12T09:54:33.463768+0000","last_undegraded":"2023-12-12T09:54:33.463768+0000","last_fullsized":"2023-12-12T09:54:33.463768+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:54:26.808349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453906+0000","last_change":"2023-12-12T09:54:33.451525+0000","last_active":"2023-12-12T09:54:33.453906+0000","last_peered":"2023-12-12T09:54:33.453906+0000","last_clean":"2023-12-12T09:54:33.453906+0000","last_became_active":"2023-12-12T09:54:33.448138+0000","last_became_peered":"2023-12-12T09:54:33.448138+0000","last_unstale":"2023-12-12T09:54:33.453906+0000","last_undegraded":"2023-12-12T09:54:33.453906+0000","last_fullsized":"2023-12-12T09:54:33.453906+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:14:27.591850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.460231+0000","last_change":"2023-12-12T09:54:33.458831+0000","last_active":"2023-12-12T09:54:33.460231+0000","last_peered":"2023-12-12T09:54:33.460231+0000","last_clean":"2023-12-12T09:54:33.460231+0000","last_became_active":"2023-12-12T09:54:33.458379+0000","last_became_peered":"2023-12-12T09:54:33.458379+0000","last_unstale":"2023-12-12T09:54:33.460231+0000","last_undegraded":"2023-12-12T09:54:33.460231+0000","last_fullsized":"2023-12-12T09:54:33.460231+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:40:29.637509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.438704+0000","last_change":"2023-12-12T09:54:33.436244+0000","last_active":"2023-12-12T09:54:33.438704+0000","last_peered":"2023-12-12T09:54:33.438704+0000","last_clean":"2023-12-12T09:54:33.438704+0000","last_became_active":"2023-12-12T09:54:33.435900+0000","last_became_peered":"2023-12-12T09:54:33.435900+0000","last_unstale":"2023-12-12T09:54:33.438704+0000","last_undegraded":"2023-12-12T09:54:33.438704+0000","last_fullsized":"2023-12-12T09:54:33.438704+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:32.422515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449359+0000","last_change":"2023-12-12T09:54:33.440417+0000","last_active":"2023-12-12T09:54:33.449359+0000","last_peered":"2023-12-12T09:54:33.449359+0000","last_clean":"2023-12-12T09:54:33.449359+0000","last_became_active":"2023-12-12T09:54:33.440149+0000","last_became_peered":"2023-12-12T09:54:33.440149+0000","last_unstale":"2023-12-12T09:54:33.449359+0000","last_undegraded":"2023-12-12T09:54:33.449359+0000","last_fullsized":"2023-12-12T09:54:33.449359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:41:48.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842394+0000","last_change":"2023-12-12T09:54:33.453918+0000","last_active":"2023-12-12T09:54:34.842394+0000","last_peered":"2023-12-12T09:54:34.842394+0000","last_clean":"2023-12-12T09:54:34.842394+0000","last_became_active":"2023-12-12T09:54:33.452501+0000","last_became_peered":"2023-12-12T09:54:33.452501+0000","last_unstale":"2023-12-12T09:54:34.842394+0000","last_undegraded":"2023-12-12T09:54:34.842394+0000","last_fullsized":"2023-12-12T09:54:34.842394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:45:13.037742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.460762+0000","last_change":"2023-12-12T09:54:33.445247+0000","last_active":"2023-12-12T09:54:33.460762+0000","last_peered":"2023-12-12T09:54:33.460762+0000","last_clean":"2023-12-12T09:54:33.460762+0000","last_became_active":"2023-12-12T09:54:33.443723+0000","last_became_peered":"2023-12-12T09:54:33.443723+0000","last_unstale":"2023-12-12T09:54:33.460762+0000","last_undegraded":"2023-12-12T09:54:33.460762+0000","last_fullsized":"2023-12-12T09:54:33.460762+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:33:25.127743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448289+0000","last_change":"2023-12-12T09:54:33.453301+0000","last_active":"2023-12-12T09:54:34.448289+0000","last_peered":"2023-12-12T09:54:34.448289+0000","last_clean":"2023-12-12T09:54:34.448289+0000","last_became_active":"2023-12-12T09:54:33.452939+0000","last_became_peered":"2023-12-12T09:54:33.452939+0000","last_unstale":"2023-12-12T09:54:34.448289+0000","last_undegraded":"2023-12-12T09:54:34.448289+0000","last_fullsized":"2023-12-12T09:54:34.448289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:03.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.841387+0000","last_change":"2023-12-12T09:54:33.452118+0000","last_active":"2023-12-12T09:54:34.841387+0000","last_peered":"2023-12-12T09:54:34.841387+0000","last_clean":"2023-12-12T09:54:34.841387+0000","last_became_active":"2023-12-12T09:54:33.451052+0000","last_became_peered":"2023-12-12T09:54:33.451052+0000","last_unstale":"2023-12-12T09:54:34.841387+0000","last_undegraded":"2023-12-12T09:54:34.841387+0000","last_fullsized":"2023-12-12T09:54:34.841387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:04:17.483412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.446047+0000","last_change":"2023-12-12T09:54:32.442157+0000","last_active":"2023-12-12T09:54:33.446047+0000","last_peered":"2023-12-12T09:54:33.446047+0000","last_clean":"2023-12-12T09:54:33.446047+0000","last_became_active":"2023-12-12T09:54:32.441813+0000","last_became_peered":"2023-12-12T09:54:32.441813+0000","last_unstale":"2023-12-12T09:54:33.446047+0000","last_undegraded":"2023-12-12T09:54:33.446047+0000","last_fullsized":"2023-12-12T09:54:33.446047+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:40:28.622399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448714+0000","last_change":"2023-12-12T09:54:33.447006+0000","last_active":"2023-12-12T09:54:34.448714+0000","last_peered":"2023-12-12T09:54:34.448714+0000","last_clean":"2023-12-12T09:54:34.448714+0000","last_became_active":"2023-12-12T09:54:33.444075+0000","last_became_peered":"2023-12-12T09:54:33.444075+0000","last_unstale":"2023-12-12T09:54:34.448714+0000","last_undegraded":"2023-12-12T09:54:34.448714+0000","last_fullsized":"2023-12-12T09:54:34.448714+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:45:13.037742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.457607+0000","last_change":"2023-12-12T09:54:32.448536+0000","last_active":"2023-12-12T09:54:33.457607+0000","last_peered":"2023-12-12T09:54:33.457607+0000","last_clean":"2023-12-12T09:54:33.457607+0000","last_became_active":"2023-12-12T09:54:32.448117+0000","last_became_peered":"2023-12-12T09:54:32.448117+0000","last_unstale":"2023-12-12T09:54:33.457607+0000","last_undegraded":"2023-12-12T09:54:33.457607+0000","last_fullsized":"2023-12-12T09:54:33.457607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:33:55.675558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451798+0000","last_change":"2023-12-12T09:54:33.444330+0000","last_active":"2023-12-12T09:54:33.451798+0000","last_peered":"2023-12-12T09:54:33.451798+0000","last_clean":"2023-12-12T09:54:33.451798+0000","last_became_active":"2023-12-12T09:54:33.444016+0000","last_became_peered":"2023-12-12T09:54:33.444016+0000","last_unstale":"2023-12-12T09:54:33.451798+0000","last_undegraded":"2023-12-12T09:54:33.451798+0000","last_fullsized":"2023-12-12T09:54:33.451798+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:47.433030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451068+0000","last_change":"2023-12-12T09:54:32.446663+0000","last_active":"2023-12-12T09:54:33.451068+0000","last_peered":"2023-12-12T09:54:33.451068+0000","last_clean":"2023-12-12T09:54:33.451068+0000","last_became_active":"2023-12-12T09:54:32.445288+0000","last_became_peered":"2023-12-12T09:54:32.445288+0000","last_unstale":"2023-12-12T09:54:33.451068+0000","last_undegraded":"2023-12-12T09:54:33.451068+0000","last_fullsized":"2023-12-12T09:54:33.451068+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:33:24.112633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453812+0000","last_change":"2023-12-12T09:54:33.450915+0000","last_active":"2023-12-12T09:54:33.453812+0000","last_peered":"2023-12-12T09:54:33.453812+0000","last_clean":"2023-12-12T09:54:33.453812+0000","last_became_active":"2023-12-12T09:54:33.450523+0000","last_became_peered":"2023-12-12T09:54:33.450523+0000","last_unstale":"2023-12-12T09:54:33.453812+0000","last_undegraded":"2023-12-12T09:54:33.453812+0000","last_fullsized":"2023-12-12T09:54:33.453812+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:04:17.483412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.449520+0000","last_change":"2023-12-12T09:54:32.445526+0000","last_active":"2023-12-12T09:54:34.449520+0000","last_peered":"2023-12-12T09:54:34.449520+0000","last_clean":"2023-12-12T09:54:34.449520+0000","last_became_active":"2023-12-12T09:54:32.445271+0000","last_became_peered":"2023-12-12T09:54:32.445271+0000","last_unstale":"2023-12-12T09:54:34.449520+0000","last_undegraded":"2023-12-12T09:54:34.449520+0000","last_fullsized":"2023-12-12T09:54:34.449520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:18:14.100288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.462503+0000","last_change":"2023-12-12T09:54:33.461071+0000","last_active":"2023-12-12T09:54:33.462503+0000","last_peered":"2023-12-12T09:54:33.462503+0000","last_clean":"2023-12-12T09:54:33.462503+0000","last_became_active":"2023-12-12T09:54:33.460803+0000","last_became_peered":"2023-12-12T09:54:33.460803+0000","last_unstale":"2023-12-12T09:54:33.462503+0000","last_undegraded":"2023-12-12T09:54:33.462503+0000","last_fullsized":"2023-12-12T09:54:33.462503+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:56.972947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.460152+0000","last_change":"2023-12-12T09:54:32.431758+0000","last_active":"2023-12-12T09:54:33.460152+0000","last_peered":"2023-12-12T09:54:33.460152+0000","last_clean":"2023-12-12T09:54:33.460152+0000","last_became_active":"2023-12-12T09:54:32.431479+0000","last_became_peered":"2023-12-12T09:54:32.431479+0000","last_unstale":"2023-12-12T09:54:33.460152+0000","last_undegraded":"2023-12-12T09:54:33.460152+0000","last_fullsized":"2023-12-12T09:54:33.460152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:14:26.576740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842370+0000","last_change":"2023-12-12T09:54:33.457942+0000","last_active":"2023-12-12T09:54:34.842370+0000","last_peered":"2023-12-12T09:54:34.842370+0000","last_clean":"2023-12-12T09:54:34.842370+0000","last_became_active":"2023-12-12T09:54:33.457687+0000","last_became_peered":"2023-12-12T09:54:33.457687+0000","last_unstale":"2023-12-12T09:54:34.842370+0000","last_undegraded":"2023-12-12T09:54:34.842370+0000","last_fullsized":"2023-12-12T09:54:34.842370+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:10:40.233777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.443151+0000","last_change":"2023-12-12T09:54:32.441501+0000","last_active":"2023-12-12T09:54:33.443151+0000","last_peered":"2023-12-12T09:54:33.443151+0000","last_clean":"2023-12-12T09:54:33.443151+0000","last_became_active":"2023-12-12T09:54:32.441140+0000","last_became_peered":"2023-12-12T09:54:32.441140+0000","last_unstale":"2023-12-12T09:54:33.443151+0000","last_undegraded":"2023-12-12T09:54:33.443151+0000","last_fullsized":"2023-12-12T09:54:33.443151+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:20.188945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451627+0000","last_change":"2023-12-12T09:54:33.447877+0000","last_active":"2023-12-12T09:54:33.451627+0000","last_peered":"2023-12-12T09:54:33.451627+0000","last_clean":"2023-12-12T09:54:33.451627+0000","last_became_active":"2023-12-12T09:54:33.447564+0000","last_became_peered":"2023-12-12T09:54:33.447564+0000","last_unstale":"2023-12-12T09:54:33.451627+0000","last_undegraded":"2023-12-12T09:54:33.451627+0000","last_fullsized":"2023-12-12T09:54:33.451627+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:24.157458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.425593+0000","last_change":"2023-12-12T09:54:32.440483+0000","last_active":"2023-12-12T09:54:33.425593+0000","last_peered":"2023-12-12T09:54:33.425593+0000","last_clean":"2023-12-12T09:54:33.425593+0000","last_became_active":"2023-12-12T09:54:32.440185+0000","last_became_peered":"2023-12-12T09:54:32.440185+0000","last_unstale":"2023-12-12T09:54:33.425593+0000","last_undegraded":"2023-12-12T09:54:33.425593+0000","last_fullsized":"2023-12-12T09:54:33.425593+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:29:55.957837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842423+0000","last_change":"2023-12-12T09:54:33.460290+0000","last_active":"2023-12-12T09:54:34.842423+0000","last_peered":"2023-12-12T09:54:34.842423+0000","last_clean":"2023-12-12T09:54:34.842423+0000","last_became_active":"2023-12-12T09:54:33.460020+0000","last_became_peered":"2023-12-12T09:54:33.460020+0000","last_unstale":"2023-12-12T09:54:34.842423+0000","last_undegraded":"2023-12-12T09:54:34.842423+0000","last_fullsized":"2023-12-12T09:54:34.842423+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:54.293178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.425722+0000","last_change":"2023-12-12T09:54:32.440839+0000","last_active":"2023-12-12T09:54:33.425722+0000","last_peered":"2023-12-12T09:54:33.425722+0000","last_clean":"2023-12-12T09:54:33.425722+0000","last_became_active":"2023-12-12T09:54:32.440502+0000","last_became_peered":"2023-12-12T09:54:32.440502+0000","last_unstale":"2023-12-12T09:54:33.425722+0000","last_undegraded":"2023-12-12T09:54:33.425722+0000","last_fullsized":"2023-12-12T09:54:33.425722+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:25:35.059090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.454421+0000","last_change":"2023-12-12T09:54:33.450701+0000","last_active":"2023-12-12T09:54:33.454421+0000","last_peered":"2023-12-12T09:54:33.454421+0000","last_clean":"2023-12-12T09:54:33.454421+0000","last_became_active":"2023-12-12T09:54:33.450346+0000","last_became_peered":"2023-12-12T09:54:33.450346+0000","last_unstale":"2023-12-12T09:54:33.454421+0000","last_undegraded":"2023-12-12T09:54:33.454421+0000","last_fullsized":"2023-12-12T09:54:33.454421+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:18:52.128679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842631+0000","last_change":"2023-12-12T09:54:32.440358+0000","last_active":"2023-12-12T09:54:34.842631+0000","last_peered":"2023-12-12T09:54:34.842631+0000","last_clean":"2023-12-12T09:54:34.842631+0000","last_became_active":"2023-12-12T09:54:32.439955+0000","last_became_peered":"2023-12-12T09:54:32.439955+0000","last_unstale":"2023-12-12T09:54:34.842631+0000","last_undegraded":"2023-12-12T09:54:34.842631+0000","last_fullsized":"2023-12-12T09:54:34.842631+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:27:58.029668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448282+0000","last_change":"2023-12-12T09:54:33.448989+0000","last_active":"2023-12-12T09:54:34.448282+0000","last_peered":"2023-12-12T09:54:34.448282+0000","last_clean":"2023-12-12T09:54:34.448282+0000","last_became_active":"2023-12-12T09:54:33.448732+0000","last_became_peered":"2023-12-12T09:54:33.448732+0000","last_unstale":"2023-12-12T09:54:34.448282+0000","last_undegraded":"2023-12-12T09:54:34.448282+0000","last_fullsized":"2023-12-12T09:54:34.448282+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:04:17.483412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.432144+0000","last_change":"2023-12-12T09:54:32.440990+0000","last_active":"2023-12-12T09:54:33.432144+0000","last_peered":"2023-12-12T09:54:33.432144+0000","last_clean":"2023-12-12T09:54:33.432144+0000","last_became_active":"2023-12-12T09:54:32.440702+0000","last_became_peered":"2023-12-12T09:54:32.440702+0000","last_unstale":"2023-12-12T09:54:33.432144+0000","last_undegraded":"2023-12-12T09:54:33.432144+0000","last_fullsized":"2023-12-12T09:54:33.432144+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:31.407405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449821+0000","last_change":"2023-12-12T09:54:33.436001+0000","last_active":"2023-12-12T09:54:33.449821+0000","last_peered":"2023-12-12T09:54:33.449821+0000","last_clean":"2023-12-12T09:54:33.449821+0000","last_became_active":"2023-12-12T09:54:33.434079+0000","last_became_peered":"2023-12-12T09:54:33.434079+0000","last_unstale":"2023-12-12T09:54:33.449821+0000","last_undegraded":"2023-12-12T09:54:33.449821+0000","last_fullsized":"2023-12-12T09:54:33.449821+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:33:56.690667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842532+0000","last_change":"2023-12-12T09:54:32.443678+0000","last_active":"2023-12-12T09:54:34.842532+0000","last_peered":"2023-12-12T09:54:34.842532+0000","last_clean":"2023-12-12T09:54:34.842532+0000","last_became_active":"2023-12-12T09:54:32.443383+0000","last_became_peered":"2023-12-12T09:54:32.443383+0000","last_unstale":"2023-12-12T09:54:34.842532+0000","last_undegraded":"2023-12-12T09:54:34.842532+0000","last_fullsized":"2023-12-12T09:54:34.842532+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:33:55.675558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.454024+0000","last_change":"2023-12-12T09:54:33.448194+0000","last_active":"2023-12-12T09:54:33.454024+0000","last_peered":"2023-12-12T09:54:33.454024+0000","last_clean":"2023-12-12T09:54:33.454024+0000","last_became_active":"2023-12-12T09:54:33.447874+0000","last_became_peered":"2023-12-12T09:54:33.447874+0000","last_unstale":"2023-12-12T09:54:33.454024+0000","last_undegraded":"2023-12-12T09:54:33.454024+0000","last_fullsized":"2023-12-12T09:54:33.454024+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:28:27.268634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.466039+0000","last_change":"2023-12-12T09:54:32.444522+0000","last_active":"2023-12-12T09:54:33.466039+0000","last_peered":"2023-12-12T09:54:33.466039+0000","last_clean":"2023-12-12T09:54:33.466039+0000","last_became_active":"2023-12-12T09:54:32.444253+0000","last_became_peered":"2023-12-12T09:54:32.444253+0000","last_unstale":"2023-12-12T09:54:33.466039+0000","last_undegraded":"2023-12-12T09:54:33.466039+0000","last_fullsized":"2023-12-12T09:54:33.466039+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:43:03.013525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448716+0000","last_change":"2023-12-12T09:54:33.445430+0000","last_active":"2023-12-12T09:54:34.448716+0000","last_peered":"2023-12-12T09:54:34.448716+0000","last_clean":"2023-12-12T09:54:34.448716+0000","last_became_active":"2023-12-12T09:54:33.445063+0000","last_became_peered":"2023-12-12T09:54:33.445063+0000","last_unstale":"2023-12-12T09:54:34.448716+0000","last_undegraded":"2023-12-12T09:54:34.448716+0000","last_fullsized":"2023-12-12T09:54:34.448716+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:54.293178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453658+0000","last_change":"2023-12-12T09:54:32.441630+0000","last_active":"2023-12-12T09:54:33.453658+0000","last_peered":"2023-12-12T09:54:33.453658+0000","last_clean":"2023-12-12T09:54:33.453658+0000","last_became_active":"2023-12-12T09:54:32.441384+0000","last_became_peered":"2023-12-12T09:54:32.441384+0000","last_unstale":"2023-12-12T09:54:33.453658+0000","last_undegraded":"2023-12-12T09:54:33.453658+0000","last_fullsized":"2023-12-12T09:54:33.453658+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:44:16.374018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.460216+0000","last_change":"2023-12-12T09:54:33.449161+0000","last_active":"2023-12-12T09:54:33.460216+0000","last_peered":"2023-12-12T09:54:33.460216+0000","last_clean":"2023-12-12T09:54:33.460216+0000","last_became_active":"2023-12-12T09:54:33.445046+0000","last_became_peered":"2023-12-12T09:54:33.445046+0000","last_unstale":"2023-12-12T09:54:33.460216+0000","last_undegraded":"2023-12-12T09:54:33.460216+0000","last_fullsized":"2023-12-12T09:54:33.460216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:01:17.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.426742+0000","last_change":"2023-12-12T09:54:32.441231+0000","last_active":"2023-12-12T09:54:33.426742+0000","last_peered":"2023-12-12T09:54:33.426742+0000","last_clean":"2023-12-12T09:54:33.426742+0000","last_became_active":"2023-12-12T09:54:32.440922+0000","last_became_peered":"2023-12-12T09:54:32.440922+0000","last_unstale":"2023-12-12T09:54:33.426742+0000","last_undegraded":"2023-12-12T09:54:33.426742+0000","last_fullsized":"2023-12-12T09:54:33.426742+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:31:08.564303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.455739+0000","last_change":"2023-12-12T09:54:33.454433+0000","last_active":"2023-12-12T09:54:33.455739+0000","last_peered":"2023-12-12T09:54:33.455739+0000","last_clean":"2023-12-12T09:54:33.455739+0000","last_became_active":"2023-12-12T09:54:33.454177+0000","last_became_peered":"2023-12-12T09:54:33.454177+0000","last_unstale":"2023-12-12T09:54:33.455739+0000","last_undegraded":"2023-12-12T09:54:33.455739+0000","last_fullsized":"2023-12-12T09:54:33.455739+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:41:48.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.841998+0000","last_change":"2023-12-12T09:54:32.442404+0000","last_active":"2023-12-12T09:54:34.841998+0000","last_peered":"2023-12-12T09:54:34.841998+0000","last_clean":"2023-12-12T09:54:34.841998+0000","last_became_active":"2023-12-12T09:54:32.442112+0000","last_became_peered":"2023-12-12T09:54:32.442112+0000","last_unstale":"2023-12-12T09:54:34.841998+0000","last_undegraded":"2023-12-12T09:54:34.841998+0000","last_fullsized":"2023-12-12T09:54:34.841998+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:16:24.728525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.447552+0000","last_change":"2023-12-12T09:54:33.439390+0000","last_active":"2023-12-12T09:54:33.447552+0000","last_peered":"2023-12-12T09:54:33.447552+0000","last_clean":"2023-12-12T09:54:33.447552+0000","last_became_active":"2023-12-12T09:54:33.439053+0000","last_became_peered":"2023-12-12T09:54:33.439053+0000","last_unstale":"2023-12-12T09:54:33.447552+0000","last_undegraded":"2023-12-12T09:54:33.447552+0000","last_fullsized":"2023-12-12T09:54:33.447552+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:03.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.461918+0000","last_change":"2023-12-12T09:54:32.442971+0000","last_active":"2023-12-12T09:54:33.461918+0000","last_peered":"2023-12-12T09:54:33.461918+0000","last_clean":"2023-12-12T09:54:33.461918+0000","last_became_active":"2023-12-12T09:54:32.442684+0000","last_became_peered":"2023-12-12T09:54:32.442684+0000","last_unstale":"2023-12-12T09:54:33.461918+0000","last_undegraded":"2023-12-12T09:54:33.461918+0000","last_fullsized":"2023-12-12T09:54:33.461918+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:53.278068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.457474+0000","last_change":"2023-12-12T09:54:33.455246+0000","last_active":"2023-12-12T09:54:33.457474+0000","last_peered":"2023-12-12T09:54:33.457474+0000","last_clean":"2023-12-12T09:54:33.457474+0000","last_became_active":"2023-12-12T09:54:33.454951+0000","last_became_peered":"2023-12-12T09:54:33.454951+0000","last_unstale":"2023-12-12T09:54:33.457474+0000","last_undegraded":"2023-12-12T09:54:33.457474+0000","last_fullsized":"2023-12-12T09:54:33.457474+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:28:03.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.459467+0000","last_change":"2023-12-12T09:54:32.443691+0000","last_active":"2023-12-12T09:54:33.459467+0000","last_peered":"2023-12-12T09:54:33.459467+0000","last_clean":"2023-12-12T09:54:33.459467+0000","last_became_active":"2023-12-12T09:54:32.443409+0000","last_became_peered":"2023-12-12T09:54:32.443409+0000","last_unstale":"2023-12-12T09:54:33.459467+0000","last_undegraded":"2023-12-12T09:54:33.459467+0000","last_fullsized":"2023-12-12T09:54:33.459467+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:45:12.022632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449479+0000","last_change":"2023-12-12T09:54:33.442991+0000","last_active":"2023-12-12T09:54:33.449479+0000","last_peered":"2023-12-12T09:54:33.449479+0000","last_clean":"2023-12-12T09:54:33.449479+0000","last_became_active":"2023-12-12T09:54:33.442408+0000","last_became_peered":"2023-12-12T09:54:33.442408+0000","last_unstale":"2023-12-12T09:54:33.449479+0000","last_undegraded":"2023-12-12T09:54:33.449479+0000","last_fullsized":"2023-12-12T09:54:33.449479+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:28:27.268634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.446648+0000","last_change":"2023-12-12T09:54:32.440300+0000","last_active":"2023-12-12T09:54:34.446648+0000","last_peered":"2023-12-12T09:54:34.446648+0000","last_clean":"2023-12-12T09:54:34.446648+0000","last_became_active":"2023-12-12T09:54:32.436850+0000","last_became_peered":"2023-12-12T09:54:32.436850+0000","last_unstale":"2023-12-12T09:54:34.446648+0000","last_undegraded":"2023-12-12T09:54:34.446648+0000","last_fullsized":"2023-12-12T09:54:34.446648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:20:57.343227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.462525+0000","last_change":"2023-12-12T09:54:33.461081+0000","last_active":"2023-12-12T09:54:33.462525+0000","last_peered":"2023-12-12T09:54:33.462525+0000","last_clean":"2023-12-12T09:54:33.462525+0000","last_became_active":"2023-12-12T09:54:33.460796+0000","last_became_peered":"2023-12-12T09:54:33.460796+0000","last_unstale":"2023-12-12T09:54:33.462525+0000","last_undegraded":"2023-12-12T09:54:33.462525+0000","last_fullsized":"2023-12-12T09:54:33.462525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:30:50.396953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451488+0000","last_change":"2023-12-12T09:54:32.447889+0000","last_active":"2023-12-12T09:54:33.451488+0000","last_peered":"2023-12-12T09:54:33.451488+0000","last_clean":"2023-12-12T09:54:33.451488+0000","last_became_active":"2023-12-12T09:54:32.447665+0000","last_became_peered":"2023-12-12T09:54:32.447665+0000","last_unstale":"2023-12-12T09:54:33.451488+0000","last_undegraded":"2023-12-12T09:54:33.451488+0000","last_fullsized":"2023-12-12T09:54:33.451488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:33:55.675558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.437527+0000","last_change":"2023-12-12T09:54:33.429814+0000","last_active":"2023-12-12T09:54:33.437527+0000","last_peered":"2023-12-12T09:54:33.437527+0000","last_clean":"2023-12-12T09:54:33.437527+0000","last_became_active":"2023-12-12T09:54:33.429505+0000","last_became_peered":"2023-12-12T09:54:33.429505+0000","last_unstale":"2023-12-12T09:54:33.437527+0000","last_undegraded":"2023-12-12T09:54:33.437527+0000","last_fullsized":"2023-12-12T09:54:33.437527+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:05:32.422515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.425477+0000","last_change":"2023-12-12T09:54:32.447519+0000","last_active":"2023-12-12T09:54:33.425477+0000","last_peered":"2023-12-12T09:54:33.425477+0000","last_clean":"2023-12-12T09:54:33.425477+0000","last_became_active":"2023-12-12T09:54:32.447167+0000","last_became_peered":"2023-12-12T09:54:32.447167+0000","last_unstale":"2023-12-12T09:54:33.425477+0000","last_undegraded":"2023-12-12T09:54:33.425477+0000","last_fullsized":"2023-12-12T09:54:33.425477+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:20.188945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.449451+0000","last_change":"2023-12-12T09:54:33.446867+0000","last_active":"2023-12-12T09:54:34.449451+0000","last_peered":"2023-12-12T09:54:34.449451+0000","last_clean":"2023-12-12T09:54:34.449451+0000","last_became_active":"2023-12-12T09:54:33.446606+0000","last_became_peered":"2023-12-12T09:54:33.446606+0000","last_unstale":"2023-12-12T09:54:34.449451+0000","last_undegraded":"2023-12-12T09:54:34.449451+0000","last_fullsized":"2023-12-12T09:54:34.449451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:14:27.591850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.456241+0000","last_change":"2023-12-12T09:54:32.443527+0000","last_active":"2023-12-12T09:54:33.456241+0000","last_peered":"2023-12-12T09:54:33.456241+0000","last_clean":"2023-12-12T09:54:33.456241+0000","last_became_active":"2023-12-12T09:54:32.442564+0000","last_became_peered":"2023-12-12T09:54:32.442564+0000","last_unstale":"2023-12-12T09:54:33.456241+0000","last_undegraded":"2023-12-12T09:54:33.456241+0000","last_fullsized":"2023-12-12T09:54:33.456241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:31:08.564303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.451814+0000","last_change":"2023-12-12T09:54:33.444637+0000","last_active":"2023-12-12T09:54:33.451814+0000","last_peered":"2023-12-12T09:54:33.451814+0000","last_clean":"2023-12-12T09:54:33.451814+0000","last_became_active":"2023-12-12T09:54:33.444346+0000","last_became_peered":"2023-12-12T09:54:33.444346+0000","last_unstale":"2023-12-12T09:54:33.451814+0000","last_undegraded":"2023-12-12T09:54:33.451814+0000","last_fullsized":"2023-12-12T09:54:33.451814+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:22:11.410125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.432357+0000","last_change":"2023-12-12T09:54:32.441702+0000","last_active":"2023-12-12T09:54:33.432357+0000","last_peered":"2023-12-12T09:54:33.432357+0000","last_clean":"2023-12-12T09:54:33.432357+0000","last_became_active":"2023-12-12T09:54:32.439279+0000","last_became_peered":"2023-12-12T09:54:32.439279+0000","last_unstale":"2023-12-12T09:54:33.432357+0000","last_undegraded":"2023-12-12T09:54:33.432357+0000","last_fullsized":"2023-12-12T09:54:33.432357+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:44:16.374018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449872+0000","last_change":"2023-12-12T09:54:33.433513+0000","last_active":"2023-12-12T09:54:33.449872+0000","last_peered":"2023-12-12T09:54:33.449872+0000","last_clean":"2023-12-12T09:54:33.449872+0000","last_became_active":"2023-12-12T09:54:33.433254+0000","last_became_peered":"2023-12-12T09:54:33.433254+0000","last_unstale":"2023-12-12T09:54:33.449872+0000","last_undegraded":"2023-12-12T09:54:33.449872+0000","last_fullsized":"2023-12-12T09:54:33.449872+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:31:09.579413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.429215+0000","last_change":"2023-12-12T09:54:32.440487+0000","last_active":"2023-12-12T09:54:33.429215+0000","last_peered":"2023-12-12T09:54:33.429215+0000","last_clean":"2023-12-12T09:54:33.429215+0000","last_became_active":"2023-12-12T09:54:32.438147+0000","last_became_peered":"2023-12-12T09:54:32.438147+0000","last_unstale":"2023-12-12T09:54:33.429215+0000","last_undegraded":"2023-12-12T09:54:33.429215+0000","last_fullsized":"2023-12-12T09:54:33.429215+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:27:53.278068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.456243+0000","last_change":"2023-12-12T09:54:33.451587+0000","last_active":"2023-12-12T09:54:33.456243+0000","last_peered":"2023-12-12T09:54:33.456243+0000","last_clean":"2023-12-12T09:54:33.456243+0000","last_became_active":"2023-12-12T09:54:33.450261+0000","last_became_peered":"2023-12-12T09:54:33.450261+0000","last_unstale":"2023-12-12T09:54:33.456243+0000","last_undegraded":"2023-12-12T09:54:33.456243+0000","last_fullsized":"2023-12-12T09:54:33.456243+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:18:52.128679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.445716+0000","last_change":"2023-12-12T09:54:32.441928+0000","last_active":"2023-12-12T09:54:34.445716+0000","last_peered":"2023-12-12T09:54:34.445716+0000","last_clean":"2023-12-12T09:54:34.445716+0000","last_became_active":"2023-12-12T09:54:32.441630+0000","last_became_peered":"2023-12-12T09:54:32.441630+0000","last_unstale":"2023-12-12T09:54:34.445716+0000","last_undegraded":"2023-12-12T09:54:34.445716+0000","last_fullsized":"2023-12-12T09:54:34.445716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:55:02.941021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448308+0000","last_change":"2023-12-12T09:54:33.453387+0000","last_active":"2023-12-12T09:54:34.448308+0000","last_peered":"2023-12-12T09:54:34.448308+0000","last_clean":"2023-12-12T09:54:34.448308+0000","last_became_active":"2023-12-12T09:54:33.453014+0000","last_became_peered":"2023-12-12T09:54:33.453014+0000","last_unstale":"2023-12-12T09:54:34.448308+0000","last_undegraded":"2023-12-12T09:54:34.448308+0000","last_fullsized":"2023-12-12T09:54:34.448308+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:10:40.233777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.447257+0000","last_change":"2023-12-12T09:54:32.446155+0000","last_active":"2023-12-12T09:54:33.447257+0000","last_peered":"2023-12-12T09:54:33.447257+0000","last_clean":"2023-12-12T09:54:33.447257+0000","last_became_active":"2023-12-12T09:54:32.445951+0000","last_became_peered":"2023-12-12T09:54:32.445951+0000","last_unstale":"2023-12-12T09:54:33.447257+0000","last_undegraded":"2023-12-12T09:54:33.447257+0000","last_fullsized":"2023-12-12T09:54:33.447257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:33:24.112633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449450+0000","last_change":"2023-12-12T09:54:33.437803+0000","last_active":"2023-12-12T09:54:33.449450+0000","last_peered":"2023-12-12T09:54:33.449450+0000","last_clean":"2023-12-12T09:54:33.449450+0000","last_became_active":"2023-12-12T09:54:33.436748+0000","last_became_peered":"2023-12-12T09:54:33.436748+0000","last_unstale":"2023-12-12T09:54:33.449450+0000","last_undegraded":"2023-12-12T09:54:33.449450+0000","last_fullsized":"2023-12-12T09:54:33.449450+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:01:21.204054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.842736+0000","last_change":"2023-12-12T09:54:32.443678+0000","last_active":"2023-12-12T09:54:34.842736+0000","last_peered":"2023-12-12T09:54:34.842736+0000","last_clean":"2023-12-12T09:54:34.842736+0000","last_became_active":"2023-12-12T09:54:32.443425+0000","last_became_peered":"2023-12-12T09:54:32.443425+0000","last_unstale":"2023-12-12T09:54:34.842736+0000","last_undegraded":"2023-12-12T09:54:34.842736+0000","last_fullsized":"2023-12-12T09:54:34.842736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:00:21.640709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.453793+0000","last_change":"2023-12-12T09:54:33.450758+0000","last_active":"2023-12-12T09:54:33.453793+0000","last_peered":"2023-12-12T09:54:33.453793+0000","last_clean":"2023-12-12T09:54:33.453793+0000","last_became_active":"2023-12-12T09:54:33.450255+0000","last_became_peered":"2023-12-12T09:54:33.450255+0000","last_unstale":"2023-12-12T09:54:33.453793+0000","last_undegraded":"2023-12-12T09:54:33.453793+0000","last_fullsized":"2023-12-12T09:54:33.453793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:54:26.808349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.427358+0000","last_change":"2023-12-12T09:54:32.438932+0000","last_active":"2023-12-12T09:54:33.427358+0000","last_peered":"2023-12-12T09:54:33.427358+0000","last_clean":"2023-12-12T09:54:33.427358+0000","last_became_active":"2023-12-12T09:54:32.438676+0000","last_became_peered":"2023-12-12T09:54:32.438676+0000","last_unstale":"2023-12-12T09:54:33.427358+0000","last_undegraded":"2023-12-12T09:54:33.427358+0000","last_fullsized":"2023-12-12T09:54:33.427358+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:04:16.468303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.462691+0000","last_change":"2023-12-12T09:54:33.452142+0000","last_active":"2023-12-12T09:54:33.462691+0000","last_peered":"2023-12-12T09:54:33.462691+0000","last_clean":"2023-12-12T09:54:33.462691+0000","last_became_active":"2023-12-12T09:54:33.451867+0000","last_became_peered":"2023-12-12T09:54:33.451867+0000","last_unstale":"2023-12-12T09:54:33.462691+0000","last_undegraded":"2023-12-12T09:54:33.462691+0000","last_fullsized":"2023-12-12T09:54:33.462691+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:18:15.115398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.430534+0000","last_change":"2023-12-12T09:54:32.444994+0000","last_active":"2023-12-12T09:54:33.430534+0000","last_peered":"2023-12-12T09:54:33.430534+0000","last_clean":"2023-12-12T09:54:33.430534+0000","last_became_active":"2023-12-12T09:54:32.444575+0000","last_became_peered":"2023-12-12T09:54:32.444575+0000","last_unstale":"2023-12-12T09:54:33.430534+0000","last_undegraded":"2023-12-12T09:54:33.430534+0000","last_fullsized":"2023-12-12T09:54:33.430534+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:10:39.218668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448752+0000","last_change":"2023-12-12T09:54:33.447622+0000","last_active":"2023-12-12T09:54:34.448752+0000","last_peered":"2023-12-12T09:54:34.448752+0000","last_clean":"2023-12-12T09:54:34.448752+0000","last_became_active":"2023-12-12T09:54:33.446865+0000","last_became_peered":"2023-12-12T09:54:33.446865+0000","last_unstale":"2023-12-12T09:54:34.448752+0000","last_undegraded":"2023-12-12T09:54:34.448752+0000","last_fullsized":"2023-12-12T09:54:34.448752+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:41:48.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.441271+0000","last_change":"2023-12-12T09:54:32.446927+0000","last_active":"2023-12-12T09:54:33.441271+0000","last_peered":"2023-12-12T09:54:33.441271+0000","last_clean":"2023-12-12T09:54:33.441271+0000","last_became_active":"2023-12-12T09:54:32.446519+0000","last_became_peered":"2023-12-12T09:54:32.446519+0000","last_unstale":"2023-12-12T09:54:33.441271+0000","last_undegraded":"2023-12-12T09:54:33.441271+0000","last_fullsized":"2023-12-12T09:54:33.441271+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:50:44.580528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.449890+0000","last_change":"2023-12-12T09:54:33.436366+0000","last_active":"2023-12-12T09:54:33.449890+0000","last_peered":"2023-12-12T09:54:33.449890+0000","last_clean":"2023-12-12T09:54:33.449890+0000","last_became_active":"2023-12-12T09:54:33.436078+0000","last_became_peered":"2023-12-12T09:54:33.436078+0000","last_unstale":"2023-12-12T09:54:33.449890+0000","last_undegraded":"2023-12-12T09:54:33.449890+0000","last_fullsized":"2023-12-12T09:54:33.449890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:16:25.743635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.440140+0000","last_change":"2023-12-12T09:54:32.449896+0000","last_active":"2023-12-12T09:54:33.440140+0000","last_peered":"2023-12-12T09:54:33.440140+0000","last_clean":"2023-12-12T09:54:33.440140+0000","last_became_active":"2023-12-12T09:54:32.449681+0000","last_became_peered":"2023-12-12T09:54:32.449681+0000","last_unstale":"2023-12-12T09:54:33.440140+0000","last_undegraded":"2023-12-12T09:54:33.440140+0000","last_fullsized":"2023-12-12T09:54:33.440140+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:40:28.622399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.462883+0000","last_change":"2023-12-12T09:54:33.460842+0000","last_active":"2023-12-12T09:54:33.462883+0000","last_peered":"2023-12-12T09:54:33.462883+0000","last_clean":"2023-12-12T09:54:33.462883+0000","last_became_active":"2023-12-12T09:54:33.460493+0000","last_became_peered":"2023-12-12T09:54:33.460493+0000","last_unstale":"2023-12-12T09:54:33.462883+0000","last_undegraded":"2023-12-12T09:54:33.462883+0000","last_fullsized":"2023-12-12T09:54:33.462883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:18:12.264577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-12T09:54:34.448527+0000","last_change":"2023-12-12T09:54:32.433995+0000","last_active":"2023-12-12T09:54:34.448527+0000","last_peered":"2023-12-12T09:54:34.448527+0000","last_clean":"2023-12-12T09:54:34.448527+0000","last_became_active":"2023-12-12T09:54:32.433337+0000","last_became_peered":"2023-12-12T09:54:32.433337+0000","last_unstale":"2023-12-12T09:54:34.448527+0000","last_undegraded":"2023-12-12T09:54:34.448527+0000","last_fullsized":"2023-12-12T09:54:34.448527+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:23.142349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.437597+0000","last_change":"2023-12-12T09:54:33.430515+0000","last_active":"2023-12-12T09:54:33.437597+0000","last_peered":"2023-12-12T09:54:33.437597+0000","last_clean":"2023-12-12T09:54:33.437597+0000","last_became_active":"2023-12-12T09:54:33.429801+0000","last_became_peered":"2023-12-12T09:54:33.429801+0000","last_unstale":"2023-12-12T09:54:33.437597+0000","last_undegraded":"2023-12-12T09:54:33.437597+0000","last_fullsized":"2023-12-12T09:54:33.437597+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:32.393186+0000","last_clean_scrub_stamp":"2023-12-12T09:54:32.393186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:28:27.268634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.425187+0000","last_change":"2023-12-12T09:54:32.440055+0000","last_active":"2023-12-12T09:54:33.425187+0000","last_peered":"2023-12-12T09:54:33.425187+0000","last_clean":"2023-12-12T09:54:33.425187+0000","last_became_active":"2023-12-12T09:54:32.439758+0000","last_became_peered":"2023-12-12T09:54:32.439758+0000","last_unstale":"2023-12-12T09:54:33.425187+0000","last_undegraded":"2023-12-12T09:54:33.425187+0000","last_fullsized":"2023-12-12T09:54:33.425187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:54:31.378076+0000","last_clean_scrub_stamp":"2023-12-12T09:54:31.378076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:00:21.640709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-12T09:54:33.441517+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:54:33.441517+0000","last_peered":"2023-12-12T09:54:33.441517+0000","last_clean":"2023-12-12T09:54:33.441517+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:54:33.441517+0000","last_undegraded":"2023-12-12T09:54:33.441517+0000","last_fullsized":"2023-12-12T09:54:33.441517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":858340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0840000000000001,"15min":1.0880000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.131,"15min":1.1339999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.7110000000000001,"5min":1.855,"15min":1.855},"last":1.5349999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0189999999999999,"15min":1.018},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.4990000000000001,"5min":1.571,"15min":1.571},"last":1.036},{"interface":"front","average":{"1min":1.139,"5min":1.1439999999999999,"15min":1.145},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5720000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.96199999999999997}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.367,"15min":1.395},"min":{"1min":0.82099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.78,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.206},{"interface":"front","average":{"1min":1.153,"5min":1.421,"15min":1.4650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6379999999999999,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.2729999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.351,"15min":1.361},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.899,"5min":2.23,"15min":2.23},"last":1.581},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.397,"15min":1.4139999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9850000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.845}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.411,"15min":1.4359999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.0449999999999999,"5min":2.649,"15min":2.649},"last":1.76},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.4079999999999999,"15min":1.4279999999999999},"min":{"1min":0.71399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7649999999999999,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.4419999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.54,"15min":1.5700000000000001},"min":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.784,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.381},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5529999999999999,"15min":1.5649999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.141,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.901}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.605,"15min":1.623},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0779999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.494},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.599,"15min":1.627},"min":{"1min":0.82599999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9370000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.6299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":602112,"data_stored":261880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.054,"15min":1.046},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.202},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1060000000000001,"15min":1.1040000000000001},"min":{"1min":0.76200000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.6200000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.77100000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6579999999999999,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.887,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2749999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1399999999999999,"15min":1.1339999999999999},"min":{"1min":0.82799999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.145},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.1140000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.3280000000000001,"15min":1.3140000000000001},"min":{"1min":1.0389999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.738},{"interface":"front","average":{"1min":1.304,"5min":1.264,"15min":1.2569999999999999},"min":{"1min":0.88600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.776,"5min":1.994,"15min":1.994},"last":1.339}]},{"osd":4,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.76500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.827,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.26},{"interface":"front","average":{"1min":1.496,"5min":1.536,"15min":1.5429999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2890000000000001,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.829}]},{"osd":5,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3859999999999999,"15min":1.3779999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.98,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.45},{"interface":"front","average":{"1min":1.55,"5min":1.595,"15min":1.603},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":2.1459999999999999,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.377}]},{"osd":7,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":1.7030000000000001,"15min":1.7090000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.4119999999999999,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.885},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.52,"15min":1.5129999999999999},"min":{"1min":0.98599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.246,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.5129999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":606208,"data_stored":267848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0429999999999999,"15min":1.026},"min":{"1min":0.71799999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0750000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.228,"15min":1.2509999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5249999999999999,"5min":3.694,"15min":3.694},"last":0.78600000000000003}]},{"osd":1,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.3220000000000001,"15min":1.3480000000000001},"min":{"1min":0.76300000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7470000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.135},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0880000000000001,"15min":1.0960000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.0900000000000001,"15min":1.0760000000000001},"min":{"1min":0.91400000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.115,"15min":1.0940000000000001},"min":{"1min":0.874,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.202}]},{"osd":3,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.361,"15min":1.371},"min":{"1min":0.79100000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.74,"5min":3.944,"15min":3.944},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.298,"15min":1.2929999999999999},"min":{"1min":0.92500000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.8260000000000001,"5min":1.925,"15min":1.925},"last":1.369}]},{"osd":4,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.341,"15min":1.335},"min":{"1min":1.0209999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0150000000000001,"5min":2.536,"15min":2.536},"last":1.48},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.53,"15min":1.554},"min":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8420000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.4370000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.462,"15min":1.464},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9430000000000001,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.518},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5009999999999999,"15min":1.4990000000000001},"min":{"1min":1.0649999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.8839999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.621,"15min":1.633},"min":{"1min":1.095,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9870000000000001,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.647,"15min":1.6639999999999999},"min":{"1min":1.1040000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.048,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.6040000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":593920,"data_stored":260791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.095,"15min":1.077},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":5.5430000000000001,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":1.028},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.196,"15min":1.1160000000000001},"min":{"1min":0.66500000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":7.29,"5min":7.29,"15min":7.29},"last":1.472}]},{"osd":1,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.2290000000000001,"15min":1.171},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.3360000000000003,"5min":7.3360000000000003,"15min":7.3360000000000003},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2949999999999999,"15min":1.288},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.7530000000000001,"5min":6.7530000000000001,"15min":6.7530000000000001},"last":1.5009999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.165,"15min":1.1539999999999999},"min":{"1min":0.67700000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.891,"5min":4.891,"15min":4.891},"last":1.341},{"interface":"front","average":{"1min":1.742,"5min":1.373,"15min":1.3109999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":7.4100000000000001,"5min":7.4100000000000001,"15min":7.4100000000000001},"last":1.3600000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3200000000000001,"15min":1.306},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.337,"15min":1.347},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.7409999999999997,"5min":4.7409999999999997,"15min":4.7409999999999997},"last":1.264}]},{"osd":5,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.9379999999999999,"5min":1.502,"15min":1.429},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.508,"15min":1.49},"min":{"1min":0.872,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":6.8499999999999996,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":1.2889999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.409,"15min":1.405},"min":{"1min":0.64800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.619,"15min":1.5620000000000001},"min":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":7.3739999999999997,"5min":7.3739999999999997,"15min":7.3739999999999997},"last":0.95299999999999996}]},{"osd":7,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.7030000000000001,"5min":1.5880000000000001,"15min":1.569},"min":{"1min":0.93799999999999994,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.823,"5min":1.613,"15min":1.5780000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":6.4260000000000002,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.137}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":610304,"data_stored":262862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81200000000000006,"15min":0.80600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.879,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.484,"5min":1.577,"15min":1.577},"last":0.99399999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.88500000000000001,"15min":0.875},"min":{"1min":0.54600000000000004,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4630000000000001,"5min":1.548,"15min":1.548},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96999999999999997,"15min":0.97399999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4279999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.55}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.05,"15min":1.0640000000000001},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.6060000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.6459999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.645,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.679},{"interface":"front","average":{"1min":1.196,"5min":1.252,"15min":1.262},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6870000000000001,"5min":1.964,"15min":1.964},"last":1.5980000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2949999999999999,"15min":1.3149999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.669,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3160000000000001,"15min":1.319},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.885,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.748}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3280000000000001,"15min":1.3320000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.738,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.228},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3169999999999999,"15min":1.327},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6499999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4330000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3440000000000001,"15min":1.351},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.7949999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.387},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.4730000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":618496,"data_stored":269460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.83599999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.89400000000000002,"15min":0.91100000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2230000000000001,"5min":1.752,"15min":1.752},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.90700000000000003,"15min":0.92300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4550000000000001,"5min":1.603,"15min":1.603},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.95499999999999996,"15min":0.96999999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4019999999999999,"5min":1.633,"15min":1.633},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.89900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.1559999999999999,"15min":1.1839999999999999},"min":{"1min":0.53800000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8879999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.63200000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.2989999999999999,"15min":1.3220000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.254,"15min":1.26},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.629,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.872}]},{"osd":5,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.1870000000000001,"15min":1.2110000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.506,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3220000000000001},"min":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.645,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.92200000000000004}]},{"osd":6,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.369,"15min":1.3919999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.631,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.2010000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.038}]},{"osd":7,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.452,"15min":1.48},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6779999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.399,"15min":1.429},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1139999999999999,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.0660000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":260369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.153,"5min":1.486,"15min":1.486},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90800000000000003,"15min":0.91900000000000004},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3520000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.78300000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.879,"15min":0.875},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.579,"5min":1.712,"15min":1.712},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.99199999999999999,"15min":0.99299999999999999},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.232}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.018,"15min":1.036},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3919999999999999,"5min":1.665,"15min":1.665},"last":1.333},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":0.98999999999999999},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4570000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.123,"15min":1.1319999999999999},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4570000000000001,"5min":2.056,"15min":2.056},"last":1.262},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2929999999999999,"15min":1.3089999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":2.427,"15min":2.427},"last":1.399}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.363,"15min":1.365},"min":{"1min":1.05,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8129999999999999,"5min":3.476,"15min":3.476},"last":1.472},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.3300000000000001,"15min":1.3500000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.7010000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.5109999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4139999999999999,"15min":1.4279999999999999},"min":{"1min":0.94699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7230000000000001,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.625},{"interface":"front","average":{"1min":1.198,"5min":1.3380000000000001,"15min":1.3620000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.573,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.5960000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3839999999999999,"15min":1.3979999999999999},"min":{"1min":0.83199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.694,"5min":3.052,"15min":3.052},"last":1.556},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4379999999999999,"15min":1.4510000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9079999999999999,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":1.3700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":850633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3999999999999999,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3420000000000001,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.56999999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.165,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0600000000000001,"15min":1.0860000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.5089999999999999,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.98199999999999998},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":1.296,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.661,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.57699999999999996}]},{"osd":4,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1779999999999999,"15min":1.1890000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.577,"5min":2.738,"15min":2.738},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.171,"5min":1.3300000000000001,"15min":1.3560000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.528,"5min":3.875,"15min":3.875},"last":1.016}]},{"osd":5,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2589999999999999,"15min":1.2729999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.625,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2749999999999999,"15min":1.2909999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5680000000000001,"5min":2.54,"15min":2.54},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.698,"5min":2.117,"15min":2.117},"last":1.002},{"interface":"front","average":{"1min":1.282,"5min":1.4179999999999999,"15min":1.4399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7230000000000001,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.80700000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3779999999999999,"15min":1.3859999999999999},"min":{"1min":0.84699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7629999999999999,"5min":2.931,"15min":2.931},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6399999999999999,"5min":4.548,"15min":4.548},"last":0.92000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:54:38.410 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-12T09:54:38.410 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-12T09:54:38.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.675+0000 7ff6e8fdb640 1 Processor -- start 2023-12-12T09:54:38.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.675+0000 7ff6e8fdb640 1 -- start start 2023-12-12T09:54:38.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.677+0000 7ff6e8fdb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff6e40b2d90 0x7ff6e415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.677+0000 7ff6e8fdb640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff6e415d510 0x7ff6e4070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.677+0000 7ff6e8fdb640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e406fff0 con 0x7ff6e40b29b0 2023-12-12T09:54:38.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.677+0000 7ff6e8fdb640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e4153ec0 con 0x7ff6e415d510 2023-12-12T09:54:38.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.677+0000 7ff6e8fdb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e415d980 con 0x7ff6e40b2d90 2023-12-12T09:54:38.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.677+0000 7ff6e37fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff6e40b29b0 0x7ff6e40b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:60556/0 (socket says 172.21.15.72:60556) 2023-12-12T09:54:38.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.677+0000 7ff6e37fe640 1 -- 172.21.15.72:0/2262282977 learned_addr learned my addr 172.21.15.72:0/2262282977 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:38.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.677+0000 7ff6e3fff640 1 --2- 172.21.15.72:0/2262282977 >> v2:172.21.15.72:3300/0 conn(0x7ff6e415d510 0x7ff6e4070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.678+0000 7ff6e2ffd640 1 --2- 172.21.15.72:0/2262282977 >> v2:172.21.15.113:3300/0 conn(0x7ff6e40b2d90 0x7ff6e415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.678+0000 7ff6e27fc640 1 -- 172.21.15.72:0/2262282977 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467965494 0 0) 0x7ff6e406fff0 con 0x7ff6e40b29b0 2023-12-12T09:54:38.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.678+0000 7ff6e3fff640 1 -- 172.21.15.72:0/2262282977 >> v2:172.21.15.113:3300/0 conn(0x7ff6e40b2d90 msgr2=0x7ff6e415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:38.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.678+0000 7ff6e3fff640 1 --2- 172.21.15.72:0/2262282977 >> v2:172.21.15.113:3300/0 conn(0x7ff6e40b2d90 0x7ff6e415cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.678+0000 7ff6e3fff640 1 -- 172.21.15.72:0/2262282977 >> v1:172.21.15.72:6789/0 conn(0x7ff6e40b29b0 legacy=0x7ff6e40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.678+0000 7ff6e3fff640 1 -- 172.21.15.72:0/2262282977 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40b4120 con 0x7ff6e415d510 2023-12-12T09:54:38.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.679+0000 7ff6e3fff640 1 --2- 172.21.15.72:0/2262282977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e415d510 0x7ff6e4070480 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40b1870 tx=0x7ff6dc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e27250fff48a9ad server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.679+0000 7ff6e27fc640 1 -- 172.21.15.72:0/2262282977 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc02cc60 con 0x7ff6e415d510 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.679+0000 7ff6e27fc640 1 -- 172.21.15.72:0/2262282977 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc02cdc0 con 0x7ff6e415d510 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.679+0000 7ff6e27fc640 1 -- 172.21.15.72:0/2262282977 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc034660 con 0x7ff6e415d510 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.680+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/2262282977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e415d510 msgr2=0x7ff6e4070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.680+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/2262282977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e415d510 0x7ff6e4070480 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40b1870 tx=0x7ff6dc02beb0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.680+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/2262282977 shutdown_connections 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.680+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/2262282977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e415d510 0x7ff6e4070480 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.680+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/2262282977 >> v2:172.21.15.113:3300/0 conn(0x7ff6e40b2d90 0x7ff6e415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.680+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/2262282977 >> 172.21.15.72:0/2262282977 conn(0x7ff6e40b0990 msgr2=0x7ff6e40b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.680+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/2262282977 shutdown_connections 2023-12-12T09:54:38.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.680+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/2262282977 wait complete. 2023-12-12T09:54:38.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.683+0000 7ff6e8fdb640 1 Processor -- start 2023-12-12T09:54:38.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.683+0000 7ff6e8fdb640 1 -- start start 2023-12-12T09:54:38.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e8fdb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b29b0 0x7ff6e414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e8fdb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e40b2d90 0x7ff6e413e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e8fdb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415d510 0x7ff6e4144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e8fdb640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40717f0 con 0x7ff6e40b2d90 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e8fdb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4071a40 con 0x7ff6e415d510 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e8fdb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4071d20 con 0x7ff6e40b29b0 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e40b2d90 0x7ff6e413e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e2ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e40b2d90 0x7ff6e413e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49892/0 (socket says 172.21.15.72:49892) 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.684+0000 7ff6e2ffd640 1 -- 172.21.15.72:0/1793865007 learned_addr learned my addr 172.21.15.72:0/1793865007 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e37fe640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b29b0 0x7ff6e414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e3fff640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415d510 0x7ff6e4144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e2ffd640 1 -- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b29b0 msgr2=0x7ff6e414ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e2ffd640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b29b0 0x7ff6e414ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e2ffd640 1 -- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415d510 msgr2=0x7ff6e4144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e2ffd640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415d510 0x7ff6e4144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e2ffd640 1 -- 172.21.15.72:0/1793865007 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc009a70 con 0x7ff6e40b2d90 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e3fff640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415d510 0x7ff6e4144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e37fe640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b29b0 0x7ff6e414ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:38.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6e2ffd640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e40b2d90 0x7ff6e413e6e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800d800 tx=0x7ff6d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:38.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6cbfff640 1 -- 172.21.15.72:0/1793865007 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800b6c0 con 0x7ff6e40b2d90 2023-12-12T09:54:38.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.685+0000 7ff6cbfff640 1 -- 172.21.15.72:0/1793865007 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d800b820 con 0x7ff6e40b2d90 2023-12-12T09:54:38.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.686+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e4143b20 con 0x7ff6e40b2d90 2023-12-12T09:54:38.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.686+0000 7ff6cbfff640 1 -- 172.21.15.72:0/1793865007 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800bbb0 con 0x7ff6e40b2d90 2023-12-12T09:54:38.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.686+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e41440e0 con 0x7ff6e40b2d90 2023-12-12T09:54:38.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.687+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e41442a0 con 0x7ff6e40b2d90 2023-12-12T09:54:38.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.688+0000 7ff6cbfff640 1 -- 172.21.15.72:0/1793865007 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6d800bd10 con 0x7ff6e40b2d90 2023-12-12T09:54:38.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.689+0000 7ff6cbfff640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff6d006ae70 0x7ff6d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:38.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.689+0000 7ff6cbfff640 1 -- 172.21.15.72:0/1793865007 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff6d8089c90 con 0x7ff6e40b2d90 2023-12-12T09:54:38.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.689+0000 7ff6e37fe640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff6d006ae70 0x7ff6d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:38.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.690+0000 7ff6e37fe640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff6d006ae70 0x7ff6d006d330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4007fd0 tx=0x7ff6d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:38.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.694+0000 7ff6cbfff640 1 -- 172.21.15.72:0/1793865007 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6d8054c00 con 0x7ff6e40b2d90 2023-12-12T09:54:38.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.951+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff6e4074450 con 0x7ff6e40b2d90 2023-12-12T09:54:38.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.957+0000 7ff6cbfff640 1 -- 172.21.15.72:0/1793865007 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff6d8058220 con 0x7ff6e40b2d90 2023-12-12T09:54:38.960 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2023-12-12T09:54:38.960 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQALLXhlQjDrNhAAcu4UiqhTEuide2bwToXy+Q== 2023-12-12T09:54:38.960 INFO:teuthology.orchestra.run.smithi072.stdout: caps mds = "allow" 2023-12-12T09:54:38.960 INFO:teuthology.orchestra.run.smithi072.stdout: caps mon = "allow r" 2023-12-12T09:54:38.960 INFO:teuthology.orchestra.run.smithi072.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-12-12T09:54:38.960 INFO:teuthology.orchestra.run.smithi072.stderr:updated caps for client.0 2023-12-12T09:54:38.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff6d006ae70 msgr2=0x7ff6d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff6d006ae70 0x7ff6d006d330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4007fd0 tx=0x7ff6d4009040 comp rx=0 tx=0).stop 2023-12-12T09:54:38.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e40b2d90 msgr2=0x7ff6e413e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:38.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e40b2d90 0x7ff6e413e6e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800d800 tx=0x7ff6d800dcc0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 shutdown_connections 2023-12-12T09:54:38.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff6d006ae70 0x7ff6d006d330 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e415d510 0x7ff6e4144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6e40b2d90 0x7ff6e413e6e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 --2- 172.21.15.72:0/1793865007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e40b29b0 0x7ff6e414ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:38.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 >> 172.21.15.72:0/1793865007 conn(0x7ff6e40b0990 msgr2=0x7ff6e406cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:38.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 shutdown_connections 2023-12-12T09:54:38.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:38.960+0000 7ff6e8fdb640 1 -- 172.21.15.72:0/1793865007 wait complete. 2023-12-12T09:54:38.979 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:39.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.189+0000 7f4edb2d3640 1 Processor -- start 2023-12-12T09:54:39.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.189+0000 7f4edb2d3640 1 -- start start 2023-12-12T09:54:39.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.190+0000 7f4edb2d3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4070170 0x7f4ed4072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:39.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.191+0000 7f4edb2d3640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed406c4b0 con 0x7f4ed415d480 2023-12-12T09:54:39.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.191+0000 7f4edb2d3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed40b2fe0 con 0x7f4ed40b3660 2023-12-12T09:54:39.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.191+0000 7f4edb2d3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed406d950 con 0x7f4ed4070170 2023-12-12T09:54:39.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.191+0000 7f4ed9870640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4ed415d480 0x7f4ed415d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:60564/0 (socket says 172.21.15.72:60564) 2023-12-12T09:54:39.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.191+0000 7f4ed9870640 1 -- 172.21.15.72:0/1408727347 learned_addr learned my addr 172.21.15.72:0/1408727347 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:39.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.191+0000 7f4ed906f640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4070170 0x7f4ed4072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:39.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.191+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835774041 0 0) 0x7f4ed406c4b0 con 0x7f4ed415d480 2023-12-12T09:54:39.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.191+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec4003520 con 0x7f4ed415d480 2023-12-12T09:54:39.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4163202165 0 0) 0x7f4ec4003520 con 0x7f4ed415d480 2023-12-12T09:54:39.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4070170 msgr2=0x7f4ed4072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:39.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4070170 0x7f4ed4072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 >> v1:172.21.15.113:6789/0 conn(0x7f4ed40b3660 legacy=0x7f4ed4070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40b2a20 con 0x7f4ed415d480 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f4ecc001670 con 0x7f4ed415d480 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 >> v1:172.21.15.72:6789/0 conn(0x7f4ed415d480 legacy=0x7f4ed415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ec4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ec40047c0 0x7f4ec4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4070170 0x7f4ec4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed406d950 con 0x7f4ec40039a0 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec400bb40 con 0x7f4ec40047c0 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec400bd00 con 0x7f4ed4070170 2023-12-12T09:54:39.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.192+0000 7f4ed906f640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ec4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.193+0000 7f4ed906f640 1 -- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4070170 msgr2=0x7f4ec4005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.193+0000 7f4ed906f640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4070170 0x7f4ec4005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.193+0000 7f4ed906f640 1 -- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ec40047c0 msgr2=0x7f4ec4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.193+0000 7f4ed906f640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ec40047c0 0x7f4ec4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.193+0000 7f4ed906f640 1 -- 172.21.15.72:0/1408727347 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4003520 con 0x7f4ec40039a0 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.194+0000 7f4ed906f640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ec4003dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009fc0 tx=0x7f4ec40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58757eadb1cb09a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.194+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec802cc60 con 0x7f4ec40039a0 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.194+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec800f040 con 0x7f4ec40039a0 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.194+0000 7f4ed3fff640 1 -- 172.21.15.72:0/1408727347 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8034440 con 0x7f4ec40039a0 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.194+0000 7f4edb2d3640 1 -- 172.21.15.72:0/1408727347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 msgr2=0x7f4ec4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.194+0000 7f4edb2d3640 1 --2- 172.21.15.72:0/1408727347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ec4003dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009fc0 tx=0x7f4ec40057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.194+0000 7f4ed9870640 1 -- 172.21.15.72:0/1408727347 reap_dead start 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.195+0000 7f4edb2d3640 1 -- 172.21.15.72:0/1408727347 shutdown_connections 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.195+0000 7f4edb2d3640 1 -- 172.21.15.72:0/1408727347 >> 172.21.15.72:0/1408727347 conn(0x7f4ed40b08d0 msgr2=0x7f4ed40b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:39.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.195+0000 7f4edb2d3640 1 -- 172.21.15.72:0/1408727347 shutdown_connections 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.195+0000 7f4edb2d3640 1 -- 172.21.15.72:0/1408727347 wait complete. 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.196+0000 7f4edb2d3640 1 Processor -- start 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.197+0000 7f4edb2d3640 1 -- start start 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.197+0000 7f4edb2d3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ed41520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.197+0000 7f4edb2d3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4156790 0x7f4ed406c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4edb2d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed4156c10 0x7f4ed406cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4edb2d3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed4158280 con 0x7f4ec40039a0 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4edb2d3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41571e0 con 0x7f4ed4156c10 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4edb2d3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41574c0 con 0x7f4ed4156790 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ed41520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ed41520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49910/0 (socket says 172.21.15.72:49910) 2023-12-12T09:54:39.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 -- 172.21.15.72:0/2441394205 learned_addr learned my addr 172.21.15.72:0/2441394205 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 -- 172.21.15.72:0/2441394205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4156790 msgr2=0x7f4ed406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4156790 0x7f4ed406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 -- 172.21.15.72:0/2441394205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed4156c10 msgr2=0x7f4ed406cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed4156c10 0x7f4ed406cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 -- 172.21.15.72:0/2441394205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed41577a0 con 0x7f4ec40039a0 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.198+0000 7f4ed906f640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ed41520a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800e230 tx=0x7f4ec8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.199+0000 7f4ed1ffb640 1 -- 172.21.15.72:0/2441394205 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8034dd0 con 0x7f4ec40039a0 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.199+0000 7f4ed1ffb640 1 -- 172.21.15.72:0/2441394205 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec8040450 con 0x7f4ec40039a0 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.199+0000 7f4ed1ffb640 1 -- 172.21.15.72:0/2441394205 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8040800 con 0x7f4ec40039a0 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.199+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed4157a80 con 0x7f4ec40039a0 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.199+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41525e0 con 0x7f4ec40039a0 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.200+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed4074450 con 0x7f4ec40039a0 2023-12-12T09:54:39.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.201+0000 7f4ed1ffb640 1 -- 172.21.15.72:0/2441394205 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ec8040960 con 0x7f4ec40039a0 2023-12-12T09:54:39.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.202+0000 7f4ed1ffb640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4eb806ae40 0x7f4eb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:39.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.202+0000 7f4ed1ffb640 1 -- 172.21.15.72:0/2441394205 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4ec80abf80 con 0x7f4ec40039a0 2023-12-12T09:54:39.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.202+0000 7f4ed886e640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4eb806ae40 0x7f4eb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:39.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.203+0000 7f4ed886e640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4eb806ae40 0x7f4eb806d300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4155990 tx=0x7f4ec0007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:39.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.206+0000 7f4ed1ffb640 1 -- 172.21.15.72:0/2441394205 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ec8076e50 con 0x7f4ec40039a0 2023-12-12T09:54:39.384 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-12-12T09:54:39.379+0000 7f84b3d41640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-12T09:54:39.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.433+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ed4067640 con 0x7f4ec40039a0 2023-12-12T09:54:39.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.434+0000 7f4ed1ffb640 1 -- 172.21.15.72:0/2441394205 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f4ec807a470 con 0x7f4ec40039a0 2023-12-12T09:54:39.436 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:39.436 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/130800926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":130800926},{"type":"v1","addr":"172.21.15.72:6836","nonce":130800926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/318582100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":318582100},{"type":"v1","addr":"172.21.15.72:6837","nonce":318582100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2380516950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2380516950},{"type":"v1","addr":"172.21.15.113:6835","nonce":2380516950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:54:32.882443+0000","modified":"2023-12-12T09:54:35.940697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/4181915308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4181915308},{"type":"v1","addr":"172.21.15.113:6834","nonce":4181915308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-12T09:54:39.436 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 25 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4eb806ae40 msgr2=0x7f4eb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4eb806ae40 0x7f4eb806d300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4155990 tx=0x7f4ec0007990 comp rx=0 tx=0).stop 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 msgr2=0x7f4ed41520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ed41520a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800e230 tx=0x7f4ec8009750 comp rx=0 tx=0).stop 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 shutdown_connections 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4eb806ae40 0x7f4eb806d300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ed4156c10 0x7f4ed406cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ed4156790 0x7f4ed406c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 --2- 172.21.15.72:0/2441394205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec40039a0 0x7f4ed41520a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:39.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.436+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 >> 172.21.15.72:0/2441394205 conn(0x7f4ed40b08d0 msgr2=0x7f4ed4070550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.437+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 shutdown_connections 2023-12-12T09:54:39.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:39.437+0000 7f4edb2d3640 1 -- 172.21.15.72:0/2441394205 wait complete. 2023-12-12T09:54:39.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-12T09:54:32.882443+0000', 'modified': '2023-12-12T09:54:35.940697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5784}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5784': {'gid': 5784, 'name': 'b', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6834/4181915308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6832', 'nonce': 4181915308}, {'type': 'v1', 'addr': '172.21.15.113:6834', 'nonce': 4181915308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-12T09:54:39.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-12T09:54:39.861 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-12T09:54:40.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.082+0000 7ff3b0000640 1 Processor -- start 2023-12-12T09:54:40.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.082+0000 7ff3b0000640 1 -- start start 2023-12-12T09:54:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.084+0000 7ff3b0000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3a81467c0 0x7ff3a80cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.084+0000 7ff3b0000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3a8146ba0 0x7ff3a80cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.084+0000 7ff3b0000640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a806d510 con 0x7ff3a806c7e0 2023-12-12T09:54:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.084+0000 7ff3b0000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a80ac860 con 0x7ff3a8146ba0 2023-12-12T09:54:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.084+0000 7ff3b0000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a806d7f0 con 0x7ff3a81467c0 2023-12-12T09:54:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.084+0000 7ff3ae59d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff3a806c7e0 0x7ff3a806cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:60572/0 (socket says 172.21.15.72:60572) 2023-12-12T09:54:40.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.084+0000 7ff3ae59d640 1 -- 172.21.15.72:0/667738833 learned_addr learned my addr 172.21.15.72:0/667738833 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3add9c640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3a81467c0 0x7ff3a80cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/667738833 >> v2:172.21.15.113:3300/0 conn(0x7ff3a8146ba0 0x7ff3a80cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656681225 0 0) 0x7ff3a806d510 con 0x7ff3a806c7e0 2023-12-12T09:54:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff394003520 con 0x7ff3a806c7e0 2023-12-12T09:54:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 702001081 0 0) 0x7ff394003520 con 0x7ff3a806c7e0 2023-12-12T09:54:40.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3a81467c0 msgr2=0x7ff3a80cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3a81467c0 0x7ff3a80cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 >> v2:172.21.15.113:3300/0 conn(0x7ff3a8146ba0 msgr2=0x7ff3a80cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 --2- 172.21.15.72:0/667738833 >> v2:172.21.15.113:3300/0 conn(0x7ff3a8146ba0 0x7ff3a80cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a806dad0 con 0x7ff3a806c7e0 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7ff3a00019f0 con 0x7ff3a806c7e0 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 >> v1:172.21.15.72:6789/0 conn(0x7ff3a806c7e0 legacy=0x7ff3a806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394003970 0x7ff394003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff394004360 0x7ff394004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff394004eb0 0x7ff39400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80ac860 con 0x7ff394004360 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a806d7f0 con 0x7ff394004eb0 2023-12-12T09:54:40.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.085+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39400bec0 con 0x7ff394003970 2023-12-12T09:54:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.086+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff394004360 0x7ff394004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.086+0000 7ff3ad59b640 1 -- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394003970 msgr2=0x7ff394003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.086+0000 7ff3add9c640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394003970 0x7ff394003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.086+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394003970 0x7ff394003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.086+0000 7ff3ad59b640 1 -- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff394004eb0 msgr2=0x7ff39400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.086+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff394004eb0 0x7ff39400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.086+0000 7ff3ad59b640 1 -- 172.21.15.72:0/667738833 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394003520 con 0x7ff394004360 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.087+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff394004360 0x7ff394004840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009a90 tx=0x7ff3940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85352aeab656bb77 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.087+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c02c6b0 con 0x7ff394004360 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.087+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c02c810 con 0x7ff394004360 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.087+0000 7ff3acd9a640 1 -- 172.21.15.72:0/667738833 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c02cba0 con 0x7ff394004360 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.088+0000 7ff3b0000640 1 -- 172.21.15.72:0/667738833 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff394004360 msgr2=0x7ff394004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.088+0000 7ff3b0000640 1 --2- 172.21.15.72:0/667738833 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff394004360 0x7ff394004840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009a90 tx=0x7ff3940057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.088+0000 7ff3ae59d640 1 -- 172.21.15.72:0/667738833 reap_dead start 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.088+0000 7ff3b0000640 1 -- 172.21.15.72:0/667738833 shutdown_connections 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.088+0000 7ff3b0000640 1 -- 172.21.15.72:0/667738833 >> 172.21.15.72:0/667738833 conn(0x7ff3a80b0a90 msgr2=0x7ff3a80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.088+0000 7ff3b0000640 1 -- 172.21.15.72:0/667738833 shutdown_connections 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.088+0000 7ff3b0000640 1 -- 172.21.15.72:0/667738833 wait complete. 2023-12-12T09:54:40.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.090+0000 7ff3b0000640 1 Processor -- start 2023-12-12T09:54:40.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.090+0000 7ff3b0000640 1 -- start start 2023-12-12T09:54:40.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3b0000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394004360 0x7ff3a813be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3b0000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a815f760 0x7ff3a815fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3b0000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3a8140e60 0x7ff3a81412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3b0000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8160120 con 0x7ff3a815f760 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3b0000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a81f3a80 con 0x7ff3a8140e60 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3b0000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8141b50 con 0x7ff394004360 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3ad59b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a815f760 0x7ff3a815fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3ad59b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a815f760 0x7ff3a815fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49932/0 (socket says 172.21.15.72:49932) 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.091+0000 7ff3ad59b640 1 -- 172.21.15.72:0/2520356377 learned_addr learned my addr 172.21.15.72:0/2520356377 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3ad59b640 1 -- 172.21.15.72:0/2520356377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394004360 msgr2=0x7ff3a813be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3add9c640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394004360 0x7ff3a813be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3ae59d640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3a8140e60 0x7ff3a81412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394004360 0x7ff3a813be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3ad59b640 1 -- 172.21.15.72:0/2520356377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3a8140e60 msgr2=0x7ff3a81412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3a8140e60 0x7ff3a81412c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3ad59b640 1 -- 172.21.15.72:0/2520356377 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a8141e30 con 0x7ff3a815f760 2023-12-12T09:54:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a815f760 0x7ff3a815fbe0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff39c037040 tx=0x7ff39c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff39a7fc640 1 -- 172.21.15.72:0/2520356377 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c003810 con 0x7ff3a815f760 2023-12-12T09:54:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff39a7fc640 1 -- 172.21.15.72:0/2520356377 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c003970 con 0x7ff3a815f760 2023-12-12T09:54:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff39a7fc640 1 -- 172.21.15.72:0/2520356377 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c003ce0 con 0x7ff3a815f760 2023-12-12T09:54:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.092+0000 7ff3b0000640 1 -- 172.21.15.72:0/2520356377 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a81f3710 con 0x7ff3a815f760 2023-12-12T09:54:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.093+0000 7ff3b0000640 1 -- 172.21.15.72:0/2520356377 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a8141800 con 0x7ff3a815f760 2023-12-12T09:54:40.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.095+0000 7ff39a7fc640 1 -- 172.21.15.72:0/2520356377 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff39c047450 con 0x7ff3a815f760 2023-12-12T09:54:40.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.096+0000 7ff39a7fc640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff38c06ae40 0x7ff38c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.096+0000 7ff39a7fc640 1 -- 172.21.15.72:0/2520356377 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff39c0abb30 con 0x7ff3a815f760 2023-12-12T09:54:40.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.096+0000 7ff3add9c640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff38c06ae40 0x7ff38c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.096+0000 7ff3b2b93740 1 Processor -- start 2023-12-12T09:54:40.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.096+0000 7ff3add9c640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff38c06ae40 0x7ff38c06d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80cc830 tx=0x7ff3a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.097+0000 7ff3b2b93740 1 -- start start 2023-12-12T09:54:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.097+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a6010 0x55d52d0a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.097+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0762c0 0x55d52d0a3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.097+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d079b90 0x55d52d0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.097+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52cfe8f40 con 0x55d52d0762c0 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.097+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52cfe6630 con 0x55d52d0a6010 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.097+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d52cfe1f20 con 0x55d52d079b90 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0762c0 0x55d52d0a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0762c0 0x55d52d0a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49942/0 (socket says 172.21.15.72:49942) 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 -- 172.21.15.72:0/3490662961 learned_addr learned my addr 172.21.15.72:0/3490662961 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 -- 172.21.15.72:0/3490662961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d079b90 msgr2=0x55d52d0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/3490662961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d079b90 0x55d52d0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 -- 172.21.15.72:0/3490662961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a6010 msgr2=0x55d52d0a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/3490662961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a6010 0x55d52d0a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 -- 172.21.15.72:0/3490662961 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d52cfa4910 con 0x55d52d0762c0 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.098+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/3490662961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0762c0 0x55d52d0a3280 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff39c076c70 tx=0x7ff39c0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a98abaf4ffccc2a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff38b7fe640 1 -- 172.21.15.72:0/3490662961 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0829c0 con 0x55d52d0762c0 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff38b7fe640 1 -- 172.21.15.72:0/3490662961 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c078b10 con 0x55d52d0762c0 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff38b7fe640 1 -- 172.21.15.72:0/3490662961 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c076e80 con 0x55d52d0762c0 2023-12-12T09:54:40.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3490662961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0762c0 msgr2=0x55d52d0a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3490662961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0762c0 0x55d52d0a3280 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff39c076c70 tx=0x7ff39c0045a0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3490662961 shutdown_connections 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3490662961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d079b90 0x55d52d0a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3490662961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0762c0 0x55d52d0a3280 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3490662961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a6010 0x55d52d0a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3490662961 >> 172.21.15.72:0/3490662961 conn(0x55d52d078530 msgr2=0x55d52d0c0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3490662961 shutdown_connections 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.099+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3490662961 wait complete. 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.100+0000 7ff3b2b93740 1 -- start start 2023-12-12T09:54:40.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.100+0000 7ff3b2b93740 1 Processor -- start 2023-12-12T09:54:40.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.101+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d079b90 0x55d52d0b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.101+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d0d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.101+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a9340 0x55d52d08ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.101+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52cf7e2f0 con 0x55d52d0a6010 2023-12-12T09:54:40.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.101+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52cf9bc70 con 0x55d52d0a9340 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.101+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d52cd44f20 con 0x55d52d079b90 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.101+0000 7ff3b2b93740 10 client.0 _ll_register_callbacks cb 0x55d52cfa03b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d0d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d0d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49958/0 (socket says 172.21.15.72:49958) 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 -- 172.21.15.72:0/2461420613 learned_addr learned my addr 172.21.15.72:0/2461420613 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 -- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d079b90 msgr2=0x55d52d0b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d079b90 0x55d52d0b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 -- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a9340 msgr2=0x55d52d08ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a9340 0x55d52d08ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 -- 172.21.15.72:0/2461420613 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a8141800 con 0x55d52d0a6010 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d0d48b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff39c000c00 tx=0x7ff39c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556cd3a961c649b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff388ff9640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff388ff9640 1 -- 172.21.15.72:0/2461420613 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0829c0 con 0x55d52d0a6010 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.102+0000 7ff388ff9640 1 -- 172.21.15.72:0/2461420613 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff39c07a9d0 con 0x55d52d0a6010 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 1 -- 172.21.15.72:0/2461420613 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d52cfdadb0 con 0x55d52d0a6010 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff388ff9640 1 -- 172.21.15.72:0/2461420613 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff39c099080 con 0x55d52d0a6010 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 10 client.5913 fetch_fsmap learned FSMap version 25 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 1 -- 172.21.15.72:0/2461420613 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55d52cfe05a0 con 0x55d52d0a6010 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff388ff9640 1 -- 172.21.15.72:0/2461420613 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d52cfdadb0 con 0x55d52d0a6010 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff388ff9640 1 -- 172.21.15.72:0/2461420613 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff39c04f070 con 0x55d52d0a6010 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 10 client.5913 fetch_fsmap finished waiting for FSMap version 25 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 10 client.5913 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 20 client.5913 populate_metadata read hostname 'smithi072' 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] conn(0x55d52d09f440 0x55d52d09f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 4 client.5913 mds_command: new command op to 5784 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:54:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.103+0000 7ff3b2b93740 1 -- 172.21.15.72:0/2461420613 --> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d52cfdaad0 con 0x55d52d09f440 2023-12-12T09:54:40.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.104+0000 7ff3add9c640 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] conn(0x55d52d09f440 0x55d52d09f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.105+0000 7ff3add9c640 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] conn(0x55d52d09f440 0x55d52d09f820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d20e0e75e9c457 server_cookie=eb8708b1ae7f1b89 in_seq=0 out_seq=0 2023-12-12T09:54:40.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.105+0000 7ff388ff9640 10 client.5913 ms_handle_connect on v2:172.21.15.113:6832/4181915308 2023-12-12T09:54:40.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff388ff9640 1 -- 172.21.15.72:0/2461420613 <== mds.0 v2:172.21.15.113:6832/4181915308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d52cfdaad0 con 0x55d52d09f440 2023-12-12T09:54:40.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff388ff9640 10 client.5913 handle_command_reply: tid=0 2023-12-12T09:54:40.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 client.5913 shutdown 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 -- 172.21.15.72:0/2461420613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 msgr2=0x55d52d0d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d0d48b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff39c000c00 tx=0x7ff39c009190 comp rx=0 tx=0).stop 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 -- 172.21.15.72:0/2461420613 shutdown_connections 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] conn(0x55d52d09f440 0x55d52d09f820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a9340 0x55d52d08ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d0d48b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff388ff9640 0 client.5913 ms_handle_reset on v2:172.21.15.113:6832/4181915308 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/2461420613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d079b90 0x55d52d0b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 -- 172.21.15.72:0/2461420613 >> 172.21.15.72:0/2461420613 conn(0x55d52d0762c0 msgr2=0x55d52d0a9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 -- 172.21.15.72:0/2461420613 shutdown_connections 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 1 -- 172.21.15.72:0/2461420613 wait complete. 2023-12-12T09:54:40.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.107+0000 7ff3b2b93740 20 client.5913 trim_cache size 0 max 16384 2023-12-12T09:54:40.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.124+0000 7ff3b2b93740 1 Processor -- start 2023-12-12T09:54:40.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.124+0000 7ff3b2b93740 1 -- start start 2023-12-12T09:54:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d079b90 0x55d52d09dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d07a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d0a9340 0x55d52d0752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52d077c70 con 0x55d52d0a6010 2023-12-12T09:54:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52d07ccf0 con 0x55d52d079b90 2023-12-12T09:54:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d52d09e620 con 0x55d52d0a9340 2023-12-12T09:54:40.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3ad59b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d07a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3ad59b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d07a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49960/0 (socket says 172.21.15.72:49960) 2023-12-12T09:54:40.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.125+0000 7ff3ad59b640 1 -- 172.21.15.72:0/3357632357 learned_addr learned my addr 172.21.15.72:0/3357632357 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:40.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff3ad59b640 1 -- 172.21.15.72:0/3357632357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d0a9340 msgr2=0x55d52d0752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/3357632357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d0a9340 0x55d52d0752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff3ad59b640 1 -- 172.21.15.72:0/3357632357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d079b90 msgr2=0x55d52d09dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/3357632357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d079b90 0x55d52d09dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff3ad59b640 1 -- 172.21.15.72:0/3357632357 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d52cf7db80 con 0x55d52d0a6010 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/3357632357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d07a160 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff39c077190 tx=0x7ff39c090000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3feca5ee938e8e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff38bfff640 1 -- 172.21.15.72:0/3357632357 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c099080 con 0x55d52d0a6010 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff38bfff640 1 -- 172.21.15.72:0/3357632357 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c07a0d0 con 0x55d52d0a6010 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.126+0000 7ff38bfff640 1 -- 172.21.15.72:0/3357632357 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c07abf0 con 0x55d52d0a6010 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3357632357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 msgr2=0x55d52d07a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3357632357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d07a160 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff39c077190 tx=0x7ff39c090000 comp rx=0 tx=0).stop 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3357632357 shutdown_connections 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3357632357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d0a9340 0x55d52d0752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3357632357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d0a6010 0x55d52d07a160 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3357632357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d079b90 0x55d52d09dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3357632357 >> 172.21.15.72:0/3357632357 conn(0x55d52d0762c0 msgr2=0x55d52d0a0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:40.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3357632357 shutdown_connections 2023-12-12T09:54:40.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3357632357 wait complete. 2023-12-12T09:54:40.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.127+0000 7ff3b2b93740 1 -- start start 2023-12-12T09:54:40.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.128+0000 7ff3b2b93740 1 Processor -- start 2023-12-12T09:54:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d079b90 0x55d52d0dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d0a6010 0x55d52d07adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3b2b93740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a9340 0x55d52d0752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52d076150 con 0x55d52d079b90 2023-12-12T09:54:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d52d0e0a90 con 0x55d52d0a9340 2023-12-12T09:54:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3b2b93740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d52d075eb0 con 0x55d52d0a6010 2023-12-12T09:54:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3b2b93740 10 client.0 _ll_register_callbacks cb 0x7ff39c092070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:54:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d079b90 0x55d52d0dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d079b90 0x55d52d0dffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49976/0 (socket says 172.21.15.72:49976) 2023-12-12T09:54:40.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 -- 172.21.15.72:0/3461304026 learned_addr learned my addr 172.21.15.72:0/3461304026 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:40.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 -- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d0a6010 msgr2=0x55d52d07adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d0a6010 0x55d52d07adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 -- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a9340 msgr2=0x55d52d0752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:40.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a9340 0x55d52d0752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 -- 172.21.15.72:0/3461304026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d52cfe05a0 con 0x55d52d079b90 2023-12-12T09:54:40.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff3add9c640 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d079b90 0x55d52d0dffa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55d52d07c7c0 tx=0x7ff3a4011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f4cfa4dd07ccb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.129+0000 7ff388ff9640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2023-12-12T09:54:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff388ff9640 1 -- 172.21.15.72:0/3461304026 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4016ae0 con 0x55d52d079b90 2023-12-12T09:54:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff388ff9640 1 -- 172.21.15.72:0/3461304026 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff3a4016c40 con 0x55d52d079b90 2023-12-12T09:54:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3461304026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff39c078b10 con 0x55d52d079b90 2023-12-12T09:54:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff388ff9640 1 -- 172.21.15.72:0/3461304026 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3a4016e00 con 0x55d52d079b90 2023-12-12T09:54:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 10 client.5919 fetch_fsmap learned FSMap version 25 2023-12-12T09:54:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3461304026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55d52cfa3170 con 0x55d52d079b90 2023-12-12T09:54:40.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff388ff9640 1 -- 172.21.15.72:0/3461304026 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c078b10 con 0x55d52d079b90 2023-12-12T09:54:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff388ff9640 1 -- 172.21.15.72:0/3461304026 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff3a40250f0 con 0x55d52d079b90 2023-12-12T09:54:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 10 client.5919 fetch_fsmap finished waiting for FSMap version 25 2023-12-12T09:54:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 10 client.5919 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:54:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 20 client.5919 populate_metadata read hostname 'smithi072' 2023-12-12T09:54:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] conn(0x55d52d075820 0x55d52d0a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 4 client.5919 mds_command: new command op to 5784 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:54:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.130+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3461304026 --> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d52cce26c0 con 0x55d52d075820 2023-12-12T09:54:40.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.131+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] conn(0x55d52d075820 0x55d52d0a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.132+0000 7ff3ad59b640 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] conn(0x55d52d075820 0x55d52d0a77b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1929ccbda9dac463 server_cookie=ec1996bd578c4f98 in_seq=0 out_seq=0 2023-12-12T09:54:40.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.132+0000 7ff388ff9640 10 client.5919 ms_handle_connect on v2:172.21.15.113:6832/4181915308 2023-12-12T09:54:40.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.132+0000 7ff388ff9640 1 -- 172.21.15.72:0/3461304026 <== mds.0 v2:172.21.15.113:6832/4181915308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d52cce26c0 con 0x55d52d075820 2023-12-12T09:54:40.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.132+0000 7ff388ff9640 10 client.5919 handle_command_reply: tid=0 2023-12-12T09:54:40.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.132+0000 7ff3b2b93740 1 client.5919 shutdown 2023-12-12T09:54:40.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3461304026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d079b90 msgr2=0x55d52d0dffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d079b90 0x55d52d0dffa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55d52d07c7c0 tx=0x7ff3a4011c80 comp rx=0 tx=0).stop 2023-12-12T09:54:40.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3461304026 shutdown_connections 2023-12-12T09:54:40.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:6832/4181915308,v1:172.21.15.113:6834/4181915308] conn(0x55d52d075820 0x55d52d0a77b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d52d0a9340 0x55d52d0752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d52d0a6010 0x55d52d07adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 --2- 172.21.15.72:0/3461304026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d52d079b90 0x55d52d0dffa0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff388ff9640 0 client.5919 ms_handle_reset on v2:172.21.15.113:6832/4181915308 2023-12-12T09:54:40.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3461304026 >> 172.21.15.72:0/3461304026 conn(0x55d52d0762c0 msgr2=0x55d52d0da9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:40.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3461304026 shutdown_connections 2023-12-12T09:54:40.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 1 -- 172.21.15.72:0/3461304026 wait complete. 2023-12-12T09:54:40.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b2b93740 20 client.5919 trim_cache size 0 max 16384 2023-12-12T09:54:40.143 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:54:40.143 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "bd07f350-42f2-4d45-9a3d-13364a8566fe", 2023-12-12T09:54:40.143 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2023-12-12T09:54:40.143 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 5784, 2023-12-12T09:54:40.144 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2023-12-12T09:54:40.144 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2023-12-12T09:54:40.144 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2023-12-12T09:54:40.144 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 6.7204740449999996, 2023-12-12T09:54:40.144 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 25, 2023-12-12T09:54:40.144 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 51, 2023-12-12T09:54:40.145 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 51, 2023-12-12T09:54:40.145 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 12.104342722 2023-12-12T09:54:40.145 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b0000640 1 -- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff38c06ae40 msgr2=0x7ff38c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b0000640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff38c06ae40 0x7ff38c06d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80cc830 tx=0x7ff3a4008040 comp rx=0 tx=0).stop 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b0000640 1 -- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a815f760 msgr2=0x7ff3a815fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.133+0000 7ff3b0000640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a815f760 0x7ff3a815fbe0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff39c037040 tx=0x7ff39c004730 comp rx=0 tx=0).stop 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.134+0000 7ff3b0000640 1 -- 172.21.15.72:0/2520356377 shutdown_connections 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.134+0000 7ff3b0000640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff38c06ae40 0x7ff38c06d300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.134+0000 7ff3b0000640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3a8140e60 0x7ff3a81412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.134+0000 7ff3b0000640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a815f760 0x7ff3a815fbe0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.134+0000 7ff3b0000640 1 --2- 172.21.15.72:0/2520356377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff394004360 0x7ff3a813be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.134+0000 7ff3b0000640 1 -- 172.21.15.72:0/2520356377 >> 172.21.15.72:0/2520356377 conn(0x7ff3a80b0a90 msgr2=0x7ff3a806cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.134+0000 7ff3b0000640 1 -- 172.21.15.72:0/2520356377 shutdown_connections 2023-12-12T09:54:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.134+0000 7ff3b0000640 1 -- 172.21.15.72:0/2520356377 wait complete. 2023-12-12T09:54:40.153 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-12T09:54:40.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.356+0000 7f364b42b640 1 Processor -- start 2023-12-12T09:54:40.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.356+0000 7f364b42b640 1 -- start start 2023-12-12T09:54:40.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.358+0000 7f364b42b640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f364415e2b0 0x7f3644164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.358+0000 7f364b42b640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364406ca40 con 0x7f364406c410 2023-12-12T09:54:40.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.358+0000 7f364b42b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3644071cd0 con 0x7f3644157a60 2023-12-12T09:54:40.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.358+0000 7f364b42b640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3644070f10 con 0x7f364415e2b0 2023-12-12T09:54:40.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.358+0000 7f36499c8640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f364415e2b0 0x7f3644164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.358+0000 7f36489c6640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f364406c410 0x7f364415dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:60582/0 (socket says 172.21.15.72:60582) 2023-12-12T09:54:40.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.358+0000 7f36489c6640 1 -- 172.21.15.72:0/2746165819 learned_addr learned my addr 172.21.15.72:0/2746165819 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:40.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.359+0000 7f36499c8640 1 -- 172.21.15.72:0/2746165819 >> v1:172.21.15.113:6789/0 conn(0x7f3644157a60 legacy=0x7f364406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.359+0000 7f36499c8640 1 -- 172.21.15.72:0/2746165819 >> v1:172.21.15.72:6789/0 conn(0x7f364406c410 legacy=0x7f364415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.359+0000 7f36499c8640 1 -- 172.21.15.72:0/2746165819 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440711f0 con 0x7f364415e2b0 2023-12-12T09:54:40.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.360+0000 7f36499c8640 1 --2- 172.21.15.72:0/2746165819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364415e2b0 0x7f3644164d00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f363c009d90 tx=0x7f363c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b395c7bf0e853f9c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.360+0000 7f363bfff640 1 -- 172.21.15.72:0/2746165819 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c02cc60 con 0x7f364415e2b0 2023-12-12T09:54:40.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.360+0000 7f363bfff640 1 -- 172.21.15.72:0/2746165819 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c02cdc0 con 0x7f364415e2b0 2023-12-12T09:54:40.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.360+0000 7f363bfff640 1 -- 172.21.15.72:0/2746165819 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c034660 con 0x7f364415e2b0 2023-12-12T09:54:40.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.361+0000 7f364b42b640 1 -- 172.21.15.72:0/2746165819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364415e2b0 msgr2=0x7f3644164d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.361+0000 7f364b42b640 1 --2- 172.21.15.72:0/2746165819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364415e2b0 0x7f3644164d00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f363c009d90 tx=0x7f363c02beb0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.361+0000 7f364b42b640 1 -- 172.21.15.72:0/2746165819 shutdown_connections 2023-12-12T09:54:40.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.361+0000 7f364b42b640 1 --2- 172.21.15.72:0/2746165819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364415e2b0 0x7f3644164d00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.361+0000 7f364b42b640 1 -- 172.21.15.72:0/2746165819 >> 172.21.15.72:0/2746165819 conn(0x7f36440b0a90 msgr2=0x7f36440b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:40.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.361+0000 7f364b42b640 1 -- 172.21.15.72:0/2746165819 shutdown_connections 2023-12-12T09:54:40.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.361+0000 7f364b42b640 1 -- 172.21.15.72:0/2746165819 wait complete. 2023-12-12T09:54:40.364 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.363+0000 7f364b42b640 1 Processor -- start 2023-12-12T09:54:40.364 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.363+0000 7f364b42b640 1 -- start start 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f364b42b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364406c410 0x7f36441487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f364b42b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644157a60 0x7f36441597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f364b42b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364415e2b0 0x7f3644159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f364b42b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3644148d20 con 0x7f3644157a60 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f364b42b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36440af480 con 0x7f364406c410 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f364b42b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36440af760 con 0x7f364415e2b0 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36489c6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644157a60 0x7f36441597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36489c6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644157a60 0x7f36441597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49994/0 (socket says 172.21.15.72:49994) 2023-12-12T09:54:40.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36489c6640 1 -- 172.21.15.72:0/2576263840 learned_addr learned my addr 172.21.15.72:0/2576263840 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:40.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36499c8640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364415e2b0 0x7f3644159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36491c7640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364406c410 0x7f36441487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36489c6640 1 -- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364415e2b0 msgr2=0x7f3644159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36489c6640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364415e2b0 0x7f3644159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36489c6640 1 -- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364406c410 msgr2=0x7f36441487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.364+0000 7f36489c6640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364406c410 0x7f36441487e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.365+0000 7f36489c6640 1 -- 172.21.15.72:0/2576263840 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009a70 con 0x7f3644157a60 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.365+0000 7f36491c7640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364406c410 0x7f36441487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.365+0000 7f36489c6640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644157a60 0x7f36441597a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f363400c4a0 tx=0x7f363400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.365+0000 7f3639ffb640 1 -- 172.21.15.72:0/2576263840 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634005ce0 con 0x7f3644157a60 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.365+0000 7f3639ffb640 1 -- 172.21.15.72:0/2576263840 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634005e40 con 0x7f3644157a60 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.365+0000 7f3639ffb640 1 -- 172.21.15.72:0/2576263840 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634007270 con 0x7f3644157a60 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.365+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36440afa40 con 0x7f3644157a60 2023-12-12T09:54:40.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.365+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36440ac900 con 0x7f3644157a60 2023-12-12T09:54:40.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.366+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36440acaf0 con 0x7f3644157a60 2023-12-12T09:54:40.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.367+0000 7f3639ffb640 1 -- 172.21.15.72:0/2576263840 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3634007490 con 0x7f3644157a60 2023-12-12T09:54:40.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.368+0000 7f3639ffb640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae40 0x7f362806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:40.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.368+0000 7f3639ffb640 1 -- 172.21.15.72:0/2576263840 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3634088cb0 con 0x7f3644157a60 2023-12-12T09:54:40.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.368+0000 7f36491c7640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae40 0x7f362806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:40.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.369+0000 7f36491c7640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae40 0x7f362806d300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3640000f60 tx=0x7f3640009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:40.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.373+0000 7f3639ffb640 1 -- 172.21.15.72:0/2576263840 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3634053c30 con 0x7f3644157a60 2023-12-12T09:54:40.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.599+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f36440745d0 con 0x7f3644157a60 2023-12-12T09:54:40.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.599+0000 7f3639ffb640 1 -- 172.21.15.72:0/2576263840 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3634017020 con 0x7f3644157a60 2023-12-12T09:54:40.601 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-12T09:54:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.601+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae40 msgr2=0x7f362806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.601+0000 7f364b42b640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae40 0x7f362806d300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3640000f60 tx=0x7f3640009040 comp rx=0 tx=0).stop 2023-12-12T09:54:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.601+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644157a60 msgr2=0x7f36441597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.601+0000 7f364b42b640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644157a60 0x7f36441597a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f363400c4a0 tx=0x7f363400c960 comp rx=0 tx=0).stop 2023-12-12T09:54:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.602+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 shutdown_connections 2023-12-12T09:54:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.602+0000 7f364b42b640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae40 0x7f362806d300 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.602+0000 7f364b42b640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364415e2b0 0x7f3644159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.602+0000 7f364b42b640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644157a60 0x7f36441597a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.602+0000 7f364b42b640 1 --2- 172.21.15.72:0/2576263840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364406c410 0x7f36441487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:40.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.602+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 >> 172.21.15.72:0/2576263840 conn(0x7f36440b0a90 msgr2=0x7f3644073030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:40.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.602+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 shutdown_connections 2023-12-12T09:54:40.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:40.602+0000 7f364b42b640 1 -- 172.21.15.72:0/2576263840 wait complete. 2023-12-12T09:54:40.621 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-12T09:54:40.621 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:54:40.622 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2023-12-12T09:54:40.622 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-12T09:54:40.622 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-12T09:54:40.622 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-12T09:54:40.622 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-12T09:54:40.622 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:54:40.622 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:54:40.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:40.623 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2023-12-12T09:54:40.676 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:54:40.676 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:54:40.676 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:54:40.676 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:54:40.676 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-12-12T09:54:40.676 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:54:40.676 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 42222sec preferred_lft 42222sec 2023-12-12T09:54:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2023-12-12T09:54:40.678 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:54:40.678 INFO:teuthology.orchestra.run.smithi072.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-12T09:54:40.678 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 3e:b1:ce:9c:47:2c brd ff:ff:ff:ff:ff:ff 2023-12-12T09:54:40.678 INFO:teuthology.orchestra.run.smithi072.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-12T09:54:40.678 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:54:40.678 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::c4c0:7aff:fe81:b145/64 scope link 2023-12-12T09:54:40.678 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:54:40.678 INFO:teuthology.orchestra.run.smithi072.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-12T09:54:40.679 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 3e:b1:ce:9c:47:2c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-12T09:54:40.679 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::3cb1:ceff:fe9c:472c/64 scope link 2023-12-12T09:54:40.679 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:54:40.679 INFO:teuthology.orchestra.run.smithi072.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-12T09:54:40.679 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 86:f9:73:7a:97:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-12T09:54:40.679 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::84f9:73ff:fe7a:975d/64 scope link 2023-12-12T09:54:40.679 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:54:40.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:40.679 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:54:40.763 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-12-12T09:54:40.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:40.767 DEBUG:teuthology.orchestra.run.smithi072:> route 2023-12-12T09:54:40.820 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2023-12-12T09:54:40.820 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:54:40.820 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-12T09:54:40.821 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-12T09:54:40.821 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:54:40.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:40.821 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-12-12T09:54:40.821 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-12-12T09:54:40.821 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-12T09:54:40.821 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-12T09:54:40.821 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-12T09:54:40.821 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-12-12T09:54:41.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:41.003 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:54:41.057 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-12T09:54:41.057 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-12T09:54:41.058 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:54:41.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:41.058 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2023-12-12T09:54:41.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:54:41.148 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-12T09:54:41.203 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:54:41.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:54:41.204 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-12T09:54:41.255 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-12T09:54:41.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:41.255 DEBUG:teuthology.orchestra.run.smithi072:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:54:41.394 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:54:41.394 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:54:41.427 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-12T09:54:41.428 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-12T09:54:41.428 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.72:3300,172.21.15.113:3300,172.21.15.113:3301" 2023-12-12T09:54:41.428 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with new device syntax: 0@bd07f350-42f2-4d45-9a3d-13364a8566fe.cephfs=/ 2023-12-12T09:54:41.428 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.72:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2023-12-12T09:54:41.429 INFO:teuthology.orchestra.run.smithi072.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-12T09:54:41.429 INFO:teuthology.orchestra.run.smithi072.stdout: You just mounted a file system that supports labels which does not 2023-12-12T09:54:41.429 INFO:teuthology.orchestra.run.smithi072.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-12T09:54:41.429 INFO:teuthology.orchestra.run.smithi072.stdout: applications will generate AVC messages and not be allowed access to 2023-12-12T09:54:41.429 INFO:teuthology.orchestra.run.smithi072.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-12T09:54:41.433 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-12T09:54:41.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:41.434 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-12T09:54:41.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:41.516 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:54:41.591 INFO:teuthology.orchestra.run.smithi072.stdout:client5089 2023-12-12T09:54:41.591 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:41.591 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:41.642 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_fragment_size_max 50 2023-12-12T09:54:41.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.859+0000 7f614fe87640 1 Processor -- start 2023-12-12T09:54:41.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.860+0000 7f614fe87640 1 -- start start 2023-12-12T09:54:41.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.861+0000 7f614fe87640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6148160b40 0x7f61480b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:41.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.861+0000 7f614fe87640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61480d0af0 con 0x7f61480b2a50 2023-12-12T09:54:41.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.861+0000 7f614fe87640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6148160f20 con 0x7f61480b35d0 2023-12-12T09:54:41.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.861+0000 7f614fe87640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61480d1520 con 0x7f6148160b40 2023-12-12T09:54:41.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.861+0000 7f614dc23640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6148160b40 0x7f61480b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:41.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.862+0000 7f614dc23640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6148160b40 0x7f61480b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:50006/0 (socket says 172.21.15.72:50006) 2023-12-12T09:54:41.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.862+0000 7f614dc23640 1 -- 172.21.15.72:0/1803741028 learned_addr learned my addr 172.21.15.72:0/1803741028 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:41.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.862+0000 7f614d422640 1 --1- 172.21.15.72:0/1803741028 >> v1:172.21.15.72:6789/0 conn(0x7f61480b2a50 0x7f61480b2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:60596/0 (socket says 172.21.15.72:60596) 2023-12-12T09:54:41.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.862+0000 7f614dc23640 1 -- 172.21.15.72:0/1803741028 >> v1:172.21.15.113:6789/0 conn(0x7f61480b35d0 legacy=0x7f61480cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:41.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.862+0000 7f614dc23640 1 -- 172.21.15.72:0/1803741028 >> v1:172.21.15.72:6789/0 conn(0x7f61480b2a50 legacy=0x7f61480b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:41.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.862+0000 7f614dc23640 1 -- 172.21.15.72:0/1803741028 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61480cf040 con 0x7f6148160b40 2023-12-12T09:54:41.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.862+0000 7f614cc21640 1 -- 172.21.15.72:0/1803741028 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770106613 0 0) 0x7f61480d0af0 con 0x7f61480b2a50 2023-12-12T09:54:41.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.863+0000 7f614dc23640 1 --2- 172.21.15.72:0/1803741028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f61480b46a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6144009870 tx=0x7f614402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8479e69e2c41d71a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:41.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.863+0000 7f614cc21640 1 -- 172.21.15.72:0/1803741028 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144030860 con 0x7f6148160b40 2023-12-12T09:54:41.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.863+0000 7f614cc21640 1 -- 172.21.15.72:0/1803741028 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61440309c0 con 0x7f6148160b40 2023-12-12T09:54:41.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.863+0000 7f614cc21640 1 -- 172.21.15.72:0/1803741028 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144030d50 con 0x7f6148160b40 2023-12-12T09:54:41.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.864+0000 7f614fe87640 1 -- 172.21.15.72:0/1803741028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 msgr2=0x7f61480b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:41.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.864+0000 7f614fe87640 1 --2- 172.21.15.72:0/1803741028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f61480b46a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6144009870 tx=0x7f614402b9a0 comp rx=0 tx=0).stop 2023-12-12T09:54:41.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.864+0000 7f614fe87640 1 -- 172.21.15.72:0/1803741028 shutdown_connections 2023-12-12T09:54:41.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.864+0000 7f614fe87640 1 --2- 172.21.15.72:0/1803741028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f61480b46a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:41.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.864+0000 7f614fe87640 1 -- 172.21.15.72:0/1803741028 >> 172.21.15.72:0/1803741028 conn(0x7f614809c420 msgr2=0x7f61480d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:41.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.864+0000 7f614fe87640 1 -- 172.21.15.72:0/1803741028 shutdown_connections 2023-12-12T09:54:41.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.864+0000 7f614fe87640 1 -- 172.21.15.72:0/1803741028 wait complete. 2023-12-12T09:54:41.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.866+0000 7f614fe87640 1 Processor -- start 2023-12-12T09:54:41.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.866+0000 7f614fe87640 1 -- start start 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614fe87640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61480b2a50 0x7f614809eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614fe87640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61480b35d0 0x7f61480b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614fe87640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f614809d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614fe87640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614809f3a0 con 0x7f6148160b40 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614fe87640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614809f680 con 0x7f61480b35d0 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614fe87640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614809f960 con 0x7f61480b2a50 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614e424640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f614809d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614e424640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f614809d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:50012/0 (socket says 172.21.15.72:50012) 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.867+0000 7f614e424640 1 -- 172.21.15.72:0/658594382 learned_addr learned my addr 172.21.15.72:0/658594382 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614d422640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61480b35d0 0x7f61480b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614dc23640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61480b2a50 0x7f614809eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614e424640 1 -- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61480b2a50 msgr2=0x7f614809eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614e424640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61480b2a50 0x7f614809eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:41.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614e424640 1 -- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61480b35d0 msgr2=0x7f61480b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:41.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614e424640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61480b35d0 0x7f61480b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:41.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614e424640 1 -- 172.21.15.72:0/658594382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440094d0 con 0x7f6148160b40 2023-12-12T09:54:41.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614dc23640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61480b2a50 0x7f614809eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:41.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614e424640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f614809d100 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f614000c780 tx=0x7f614000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:41.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f613e7fc640 1 -- 172.21.15.72:0/658594382 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140004b60 con 0x7f6148160b40 2023-12-12T09:54:41.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f613e7fc640 1 -- 172.21.15.72:0/658594382 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614000ce90 con 0x7f6148160b40 2023-12-12T09:54:41.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614809fc40 con 0x7f6148160b40 2023-12-12T09:54:41.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f613e7fc640 1 -- 172.21.15.72:0/658594382 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614000a850 con 0x7f6148160b40 2023-12-12T09:54:41.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.868+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61480b8520 con 0x7f6148160b40 2023-12-12T09:54:41.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.870+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61480a0160 con 0x7f6148160b40 2023-12-12T09:54:41.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.871+0000 7f613e7fc640 1 -- 172.21.15.72:0/658594382 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f614000a9b0 con 0x7f6148160b40 2023-12-12T09:54:41.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.871+0000 7f613e7fc640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f612c06ae40 0x7f612c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:41.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.872+0000 7f613e7fc640 1 -- 172.21.15.72:0/658594382 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f61400898c0 con 0x7f6148160b40 2023-12-12T09:54:41.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.872+0000 7f614dc23640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f612c06ae40 0x7f612c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:41.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.872+0000 7f614dc23640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f612c06ae40 0x7f612c06d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6144036000 tx=0x7f614403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:41.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:41.876+0000 7f613e7fc640 1 -- 172.21.15.72:0/658594382 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61480a0160 con 0x7f6148160b40 2023-12-12T09:54:42.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.065+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f61480a0160 con 0x7f6148160b40 2023-12-12T09:54:42.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.071+0000 7f613e7fc640 1 -- 172.21.15.72:0/658594382 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v13)=0 v13) v1 ==== 124+0+0 (secure 0 0 0) 0x7f61480a0160 con 0x7f6148160b40 2023-12-12T09:54:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f612c06ae40 msgr2=0x7f612c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f612c06ae40 0x7f612c06d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6144036000 tx=0x7f614403b040 comp rx=0 tx=0).stop 2023-12-12T09:54:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 msgr2=0x7f614809d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f614809d100 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f614000c780 tx=0x7f614000cc40 comp rx=0 tx=0).stop 2023-12-12T09:54:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 shutdown_connections 2023-12-12T09:54:42.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f612c06ae40 0x7f612c06d300 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6148160b40 0x7f614809d100 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61480b35d0 0x7f61480b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 --2- 172.21.15.72:0/658594382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61480b2a50 0x7f614809eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 >> 172.21.15.72:0/658594382 conn(0x7f614809c420 msgr2=0x7f61480d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 shutdown_connections 2023-12-12T09:54:42.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:42.074+0000 7f614fe87640 1 -- 172.21.15.72:0/658594382 wait complete. 2023-12-12T09:54:52.098 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:52.098 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> stdin-killer -- python3 -c ' 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> import os 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> import uuid 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> n = 51 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> create_hard_links = False 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> if 0 > 0: 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> create_hard_links = True 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> path = "/home/ubuntu/cephtest/mnt.0/subdir/file" 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> dpath = os.path.dirname(path) 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> fnameprefix = os.path.basename(path) 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> os.makedirs(dpath, exist_ok=True) 2023-12-12T09:54:52.159 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> try: 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> for i in range(n): 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> with open(fpath, '"'"'w'"'"') as f: 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> f.write(f"{i}") 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> if False: 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> f.flush() 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> os.fsync(f.fileno()) 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> if False: 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> os.unlink(fpath) 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> if False: 2023-12-12T09:54:52.160 DEBUG:teuthology.orchestra.run.smithi072:> os.fsync(dirfd) 2023-12-12T09:54:52.161 DEBUG:teuthology.orchestra.run.smithi072:> if create_hard_links: 2023-12-12T09:54:52.161 DEBUG:teuthology.orchestra.run.smithi072:> for j in range(0): 2023-12-12T09:54:52.161 DEBUG:teuthology.orchestra.run.smithi072:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-12-12T09:54:52.161 DEBUG:teuthology.orchestra.run.smithi072:> if True: 2023-12-12T09:54:52.161 DEBUG:teuthology.orchestra.run.smithi072:> os.fsync(dirfd) 2023-12-12T09:54:52.161 DEBUG:teuthology.orchestra.run.smithi072:> finally: 2023-12-12T09:54:52.161 DEBUG:teuthology.orchestra.run.smithi072:> os.close(dirfd) 2023-12-12T09:54:52.161 DEBUG:teuthology.orchestra.run.smithi072:> ' 2023-12-12T09:54:52.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-12T09:54:52.396 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-12T09:54:52.396 DEBUG:teuthology.orchestra.run.smithi072:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-12T09:54:52.554 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-12T09:54:52.675 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-12T09:54:52.675 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:52.675 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:52.737 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:52.737 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:52.792 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:53.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.005+0000 7fef47083640 1 Processor -- start 2023-12-12T09:54:53.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.005+0000 7fef47083640 1 -- start start 2023-12-12T09:54:53.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.006+0000 7fef47083640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fef4015d150 0x7fef40071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.006+0000 7fef47083640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef4006dd00 0x7fef40070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.006+0000 7fef47083640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef400abd20 con 0x7fef400705b0 2023-12-12T09:54:53.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.006+0000 7fef47083640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef40070a40 con 0x7fef4015d150 2023-12-12T09:54:53.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.007+0000 7fef47083640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4015d530 con 0x7fef4006dd00 2023-12-12T09:54:53.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.007+0000 7fef46882640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fef400705b0 0x7fef40160cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:38718/0 (socket says 172.21.15.72:38718) 2023-12-12T09:54:53.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.007+0000 7fef46882640 1 -- 172.21.15.72:0/2473899599 learned_addr learned my addr 172.21.15.72:0/2473899599 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:53.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.007+0000 7fef45880640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef4006dd00 0x7fef40070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.007+0000 7fef46081640 1 --2- 172.21.15.72:0/2473899599 >> v2:172.21.15.113:3300/0 conn(0x7fef4015d150 0x7fef40071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.007+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008563817 0 0) 0x7fef400abd20 con 0x7fef400705b0 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.007+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef2c003520 con 0x7fef400705b0 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 38098614 0 0) 0x7fef2c003520 con 0x7fef400705b0 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef4006dd00 msgr2=0x7fef40070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef4006dd00 0x7fef40070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 >> v2:172.21.15.113:3300/0 conn(0x7fef4015d150 msgr2=0x7fef40071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 --2- 172.21.15.72:0/2473899599 >> v2:172.21.15.113:3300/0 conn(0x7fef4015d150 0x7fef40071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400b2950 con 0x7fef400705b0 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fef380019f0 con 0x7fef400705b0 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 >> v1:172.21.15.72:6789/0 conn(0x7fef400705b0 legacy=0x7fef40160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef4006dd00 0x7fef2c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef2c004d00 0x7fef2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40070a40 con 0x7fef2c003f40 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4015d530 con 0x7fef2c004d00 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c00b980 con 0x7fef4006dd00 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.008+0000 7fef45880640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.009+0000 7fef46081640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef4006dd00 0x7fef2c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.009+0000 7fef45880640 1 -- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef4006dd00 msgr2=0x7fef2c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.009+0000 7fef45880640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef4006dd00 0x7fef2c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.009+0000 7fef45880640 1 -- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef2c004d00 msgr2=0x7fef2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.009+0000 7fef45880640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef2c004d00 0x7fef2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.009+0000 7fef45880640 1 -- 172.21.15.72:0/2473899599 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c003520 con 0x7fef2c003f40 2023-12-12T09:54:53.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.010+0000 7fef45880640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef2c0047c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fef30009ac0 tx=0x7fef2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5241c519eee0a27d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.010+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3002c680 con 0x7fef2c003f40 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.010+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3002c7e0 con 0x7fef2c003f40 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.010+0000 7fef4507f640 1 -- 172.21.15.72:0/2473899599 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3002cb70 con 0x7fef2c003f40 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.010+0000 7fef47083640 1 -- 172.21.15.72:0/2473899599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 msgr2=0x7fef2c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.010+0000 7fef47083640 1 --2- 172.21.15.72:0/2473899599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef2c0047c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fef30009ac0 tx=0x7fef2c0057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.010+0000 7fef46882640 1 -- 172.21.15.72:0/2473899599 reap_dead start 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.011+0000 7fef47083640 1 -- 172.21.15.72:0/2473899599 shutdown_connections 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.011+0000 7fef47083640 1 -- 172.21.15.72:0/2473899599 >> 172.21.15.72:0/2473899599 conn(0x7fef400b0860 msgr2=0x7fef400b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.011+0000 7fef47083640 1 -- 172.21.15.72:0/2473899599 shutdown_connections 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.011+0000 7fef47083640 1 -- 172.21.15.72:0/2473899599 wait complete. 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.012+0000 7fef47083640 1 Processor -- start 2023-12-12T09:54:53.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.013+0000 7fef47083640 1 -- start start 2023-12-12T09:54:53.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.013+0000 7fef47083640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef40144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.013+0000 7fef47083640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef400cdb10 0x7fef401471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.013+0000 7fef47083640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef400afb20 0x7fef400affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.013+0000 7fef47083640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef400b04e0 con 0x7fef2c003f40 2023-12-12T09:54:53.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.013+0000 7fef47083640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef401451b0 con 0x7fef400cdb10 2023-12-12T09:54:53.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.013+0000 7fef47083640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef40147ad0 con 0x7fef400afb20 2023-12-12T09:54:53.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef40144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef40144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57152/0 (socket says 172.21.15.72:57152) 2023-12-12T09:54:53.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 -- 172.21.15.72:0/2724941908 learned_addr learned my addr 172.21.15.72:0/2724941908 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:53.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 -- 172.21.15.72:0/2724941908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef400afb20 msgr2=0x7fef400affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef400afb20 0x7fef400affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 -- 172.21.15.72:0/2724941908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef400cdb10 msgr2=0x7fef401471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef400cdb10 0x7fef401471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 -- 172.21.15.72:0/2724941908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40147db0 con 0x7fef2c003f40 2023-12-12T09:54:53.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.014+0000 7fef46081640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef40144850 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0108e0 tx=0x7fef3c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.015+0000 7fef36ffd640 1 -- 172.21.15.72:0/2724941908 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00eb60 con 0x7fef2c003f40 2023-12-12T09:54:53.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.015+0000 7fef36ffd640 1 -- 172.21.15.72:0/2724941908 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00ecc0 con 0x7fef2c003f40 2023-12-12T09:54:53.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.015+0000 7fef36ffd640 1 -- 172.21.15.72:0/2724941908 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c01c400 con 0x7fef2c003f40 2023-12-12T09:54:53.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.015+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef40145450 con 0x7fef2c003f40 2023-12-12T09:54:53.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.015+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef40144d90 con 0x7fef2c003f40 2023-12-12T09:54:53.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.016+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4006c3b0 con 0x7fef2c003f40 2023-12-12T09:54:53.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.017+0000 7fef36ffd640 1 -- 172.21.15.72:0/2724941908 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef3c01c560 con 0x7fef2c003f40 2023-12-12T09:54:53.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.018+0000 7fef36ffd640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fef2406ae70 0x7fef2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.018+0000 7fef36ffd640 1 -- 172.21.15.72:0/2724941908 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fef3c089eb0 con 0x7fef2c003f40 2023-12-12T09:54:53.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.018+0000 7fef45880640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fef2406ae70 0x7fef2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.019+0000 7fef45880640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fef2406ae70 0x7fef2406d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fef30036000 tx=0x7fef3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.023+0000 7fef36ffd640 1 -- 172.21.15.72:0/2724941908 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef3c054e30 con 0x7fef2c003f40 2023-12-12T09:54:53.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.214+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef4006b740 con 0x7fef2c003f40 2023-12-12T09:54:53.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.217+0000 7fef36ffd640 1 -- 172.21.15.72:0/2724941908 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16774 (secure 0 0 0) 0x7fef3c058450 con 0x7fef2c003f40 2023-12-12T09:54:53.220 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:53.220 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":51,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:34.428514+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:54:30.694215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-12T09:54:31.894291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:53.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fef2406ae70 msgr2=0x7fef2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fef2406ae70 0x7fef2406d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fef30036000 tx=0x7fef3002e000 comp rx=0 tx=0).stop 2023-12-12T09:54:53.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 msgr2=0x7fef40144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef40144850 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0108e0 tx=0x7fef3c010da0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 shutdown_connections 2023-12-12T09:54:53.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fef2406ae70 0x7fef2406d330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef400afb20 0x7fef400affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef400cdb10 0x7fef401471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 --2- 172.21.15.72:0/2724941908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef2c003f40 0x7fef40144850 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 >> 172.21.15.72:0/2724941908 conn(0x7fef400b0860 msgr2=0x7fef4009c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:53.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 shutdown_connections 2023-12-12T09:54:53.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.220+0000 7fef47083640 1 -- 172.21.15.72:0/2724941908 wait complete. 2023-12-12T09:54:53.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:54:30.694215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:54:31.894291+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:54:53.242 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:54:53.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.460+0000 7fa2e9461640 1 Processor -- start 2023-12-12T09:54:53.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.460+0000 7fa2e9461640 1 -- start start 2023-12-12T09:54:53.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e9461640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2e40ce7e0 0x7fa2e415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e9461640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2e406a460 0x7fa2e406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e9461640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e4071cd0 con 0x7fa2e415af50 2023-12-12T09:54:53.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e9461640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e406ae30 con 0x7fa2e40ce7e0 2023-12-12T09:54:53.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e9461640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40717e0 con 0x7fa2e406a460 2023-12-12T09:54:53.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e2ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fa2e415af50 0x7fa2e406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:38732/0 (socket says 172.21.15.72:38732) 2023-12-12T09:54:53.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e2ffd640 1 -- 172.21.15.72:0/2326804576 learned_addr learned my addr 172.21.15.72:0/2326804576 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:53.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2dbfff640 1 --2- 172.21.15.72:0/2326804576 >> v2:172.21.15.113:3300/0 conn(0x7fa2e40ce7e0 0x7fa2e415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e37fe640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2e406a460 0x7fa2e406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.462+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759405004 0 0) 0x7fa2e4071cd0 con 0x7fa2e415af50 2023-12-12T09:54:53.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d0003520 con 0x7fa2e415af50 2023-12-12T09:54:53.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2465242132 0 0) 0x7fa2d0003520 con 0x7fa2e415af50 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2e406a460 msgr2=0x7fa2e406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2e406a460 0x7fa2e406a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 >> v2:172.21.15.113:3300/0 conn(0x7fa2e40ce7e0 msgr2=0x7fa2e415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 --2- 172.21.15.72:0/2326804576 >> v2:172.21.15.113:3300/0 conn(0x7fa2e40ce7e0 0x7fa2e415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e40719f0 con 0x7fa2e415af50 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fa2d4001670 con 0x7fa2e415af50 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 >> v1:172.21.15.72:6789/0 conn(0x7fa2e415af50 legacy=0x7fa2e406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2e406a460 0x7fa2d0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2d0003f40 0x7fa2d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2d0004d00 0x7fa2d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e406ae30 con 0x7fa2d0003f40 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40717e0 con 0x7fa2d0004d00 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d000bab0 con 0x7fa2e406a460 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.463+0000 7fa2dbfff640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2d0003f40 0x7fa2d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.464+0000 7fa2e2ffd640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2e406a460 0x7fa2d0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.464+0000 7fa2dbfff640 1 -- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2e406a460 msgr2=0x7fa2d0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.464+0000 7fa2dbfff640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2e406a460 0x7fa2d0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.464+0000 7fa2e37fe640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2d0004d00 0x7fa2d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.464+0000 7fa2dbfff640 1 -- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2d0004d00 msgr2=0x7fa2d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.464+0000 7fa2dbfff640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2d0004d00 0x7fa2d0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.464+0000 7fa2dbfff640 1 -- 172.21.15.72:0/2326804576 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0003520 con 0x7fa2d0003f40 2023-12-12T09:54:53.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.465+0000 7fa2dbfff640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2d0003f40 0x7fa2d00047c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009ac0 tx=0x7fa2d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ae6fda85abdee0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.465+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc02c710 con 0x7fa2d0003f40 2023-12-12T09:54:53.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.465+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc02c870 con 0x7fa2d0003f40 2023-12-12T09:54:53.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.465+0000 7fa2e27fc640 1 -- 172.21.15.72:0/2326804576 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc02cc00 con 0x7fa2d0003f40 2023-12-12T09:54:53.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.466+0000 7fa2e9461640 1 -- 172.21.15.72:0/2326804576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2d0003f40 msgr2=0x7fa2d00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.466+0000 7fa2e9461640 1 --2- 172.21.15.72:0/2326804576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2d0003f40 0x7fa2d00047c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009ac0 tx=0x7fa2d00057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.466+0000 7fa2e37fe640 1 -- 172.21.15.72:0/2326804576 reap_dead start 2023-12-12T09:54:53.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.466+0000 7fa2e9461640 1 -- 172.21.15.72:0/2326804576 shutdown_connections 2023-12-12T09:54:53.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.466+0000 7fa2e9461640 1 -- 172.21.15.72:0/2326804576 >> 172.21.15.72:0/2326804576 conn(0x7fa2e40cee70 msgr2=0x7fa2e40cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:53.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.466+0000 7fa2e9461640 1 -- 172.21.15.72:0/2326804576 shutdown_connections 2023-12-12T09:54:53.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.466+0000 7fa2e9461640 1 -- 172.21.15.72:0/2326804576 wait complete. 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.468+0000 7fa2e9461640 1 Processor -- start 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.468+0000 7fa2e9461640 1 -- start start 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e9461640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2d0003f40 0x7fa2e40999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e9461640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e406c410 0x7fa2e40c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e9461640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2e406c890 0x7fa2e40c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e9461640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40c3c10 con 0x7fa2e406c890 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e9461640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e40c4190 con 0x7fa2e406c410 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e9461640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e409c940 con 0x7fa2d0003f40 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2e406c890 0x7fa2e40c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2e406c890 0x7fa2e40c2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57170/0 (socket says 172.21.15.72:57170) 2023-12-12T09:54:53.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e37fe640 1 -- 172.21.15.72:0/142897246 learned_addr learned my addr 172.21.15.72:0/142897246 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:53.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e37fe640 1 -- 172.21.15.72:0/142897246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2d0003f40 msgr2=0x7fa2e40999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e37fe640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2d0003f40 0x7fa2e40999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e37fe640 1 -- 172.21.15.72:0/142897246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e406c410 msgr2=0x7fa2e40c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e37fe640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e406c410 0x7fa2e40c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.469+0000 7fa2e37fe640 1 -- 172.21.15.72:0/142897246 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e409cc20 con 0x7fa2e406c890 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.470+0000 7fa2e37fe640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2e406c890 0x7fa2e40c2b50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40b1830 tx=0x7fa2dc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.470+0000 7fa2db7fe640 1 -- 172.21.15.72:0/142897246 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc017400 con 0x7fa2e406c890 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.470+0000 7fa2db7fe640 1 -- 172.21.15.72:0/142897246 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc011040 con 0x7fa2e406c890 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.470+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e40c3e20 con 0x7fa2e406c890 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.470+0000 7fa2db7fe640 1 -- 172.21.15.72:0/142897246 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0177c0 con 0x7fa2e406c890 2023-12-12T09:54:53.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.470+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e4099f30 con 0x7fa2e406c890 2023-12-12T09:54:53.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.471+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e409cdb0 con 0x7fa2e406c890 2023-12-12T09:54:53.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.472+0000 7fa2db7fe640 1 -- 172.21.15.72:0/142897246 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2dc017920 con 0x7fa2e406c890 2023-12-12T09:54:53.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.473+0000 7fa2db7fe640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa2c406ae40 0x7fa2c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.473+0000 7fa2db7fe640 1 -- 172.21.15.72:0/142897246 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa2dc089c70 con 0x7fa2e406c890 2023-12-12T09:54:53.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.473+0000 7fa2e2ffd640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa2c406ae40 0x7fa2c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.474+0000 7fa2e2ffd640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa2c406ae40 0x7fa2c406d300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4001200 tx=0x7fa2d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.478+0000 7fa2db7fe640 1 -- 172.21.15.72:0/142897246 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2dc054b40 con 0x7fa2e406c890 2023-12-12T09:54:53.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.672+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2e4074450 con 0x7fa2e406c890 2023-12-12T09:54:53.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.673+0000 7fa2db7fe640 1 -- 172.21.15.72:0/142897246 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2dc01c0a0 con 0x7fa2e406c890 2023-12-12T09:54:53.675 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:54:53.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa2c406ae40 msgr2=0x7fa2c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa2c406ae40 0x7fa2c406d300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4001200 tx=0x7fa2d4009040 comp rx=0 tx=0).stop 2023-12-12T09:54:53.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2e406c890 msgr2=0x7fa2e40c2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2e406c890 0x7fa2e40c2b50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40b1830 tx=0x7fa2dc00f4a0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 shutdown_connections 2023-12-12T09:54:53.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fa2c406ae40 0x7fa2c406d300 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2e406c890 0x7fa2e40c2b50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2e406c410 0x7fa2e40c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 --2- 172.21.15.72:0/142897246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2d0003f40 0x7fa2e40999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 >> 172.21.15.72:0/142897246 conn(0x7fa2e40cee70 msgr2=0x7fa2e415b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:53.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 shutdown_connections 2023-12-12T09:54:53.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.676+0000 7fa2e9461640 1 -- 172.21.15.72:0/142897246 wait complete. 2023-12-12T09:54:53.697 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:54:53.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.904+0000 7f80b9c3d640 1 Processor -- start 2023-12-12T09:54:53.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.904+0000 7f80b9c3d640 1 -- start start 2023-12-12T09:54:53.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.906+0000 7f80b9c3d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.906+0000 7f80b9c3d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80b41432b0 0x7f80b406d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.906+0000 7f80b9c3d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b40b4980 con 0x7f80b406da40 2023-12-12T09:54:53.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.906+0000 7f80b9c3d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b406ded0 con 0x7f80b41432b0 2023-12-12T09:54:53.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.906+0000 7f80b9c3d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40ce2a0 con 0x7f80b4142de0 2023-12-12T09:54:53.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.906+0000 7f80b37fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b2ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80b41432b0 0x7f80b406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b2ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80b41432b0 0x7f80b406d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:60616/0 (socket says 172.21.15.72:60616) 2023-12-12T09:54:53.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b2ffd640 1 -- 172.21.15.72:0/3352659643 learned_addr learned my addr 172.21.15.72:0/3352659643 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:53.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b3fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f80b406da40 0x7f80b415dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:51536/0 (socket says 172.21.15.72:51536) 2023-12-12T09:54:53.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b2ffd640 1 -- 172.21.15.72:0/3352659643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 msgr2=0x7f80b40cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:53.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b2ffd640 1 --2- 172.21.15.72:0/3352659643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b2ffd640 1 -- 172.21.15.72:0/3352659643 >> v1:172.21.15.113:6789/0 conn(0x7f80b406da40 legacy=0x7f80b415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b2ffd640 1 -- 172.21.15.72:0/3352659643 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4071260 con 0x7f80b41432b0 2023-12-12T09:54:53.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.907+0000 7f80b37fe640 1 --2- 172.21.15.72:0/3352659643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:54:53.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.909+0000 7f80b2ffd640 1 --2- 172.21.15.72:0/3352659643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b41432b0 0x7f80b406d500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009870 tx=0x7f80a802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=203efbec8d1f7c3d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.909+0000 7f80b27fc640 1 -- 172.21.15.72:0/3352659643 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8031640 con 0x7f80b41432b0 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.909+0000 7f80b27fc640 1 -- 172.21.15.72:0/3352659643 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a80317a0 con 0x7f80b41432b0 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.909+0000 7f80b27fc640 1 -- 172.21.15.72:0/3352659643 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8031b30 con 0x7f80b41432b0 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.910+0000 7f80b9c3d640 1 -- 172.21.15.72:0/3352659643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b41432b0 msgr2=0x7f80b406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.910+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/3352659643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b41432b0 0x7f80b406d500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009870 tx=0x7f80a802b8f0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.910+0000 7f80b9c3d640 1 -- 172.21.15.72:0/3352659643 shutdown_connections 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.910+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/3352659643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b41432b0 0x7f80b406d500 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.910+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/3352659643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.910+0000 7f80b9c3d640 1 -- 172.21.15.72:0/3352659643 >> 172.21.15.72:0/3352659643 conn(0x7f80b40cb850 msgr2=0x7f80b40cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.910+0000 7f80b9c3d640 1 -- 172.21.15.72:0/3352659643 shutdown_connections 2023-12-12T09:54:53.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.910+0000 7f80b9c3d640 1 -- 172.21.15.72:0/3352659643 wait complete. 2023-12-12T09:54:53.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.911+0000 7f80b9c3d640 1 Processor -- start 2023-12-12T09:54:53.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.912+0000 7f80b9c3d640 1 -- start start 2023-12-12T09:54:53.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.912+0000 7f80b9c3d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b406da40 0x7f80b4159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.912+0000 7f80b9c3d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.912+0000 7f80b9c3d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b41432b0 0x7f80b40a8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.912+0000 7f80b9c3d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b413e0b0 con 0x7f80b41432b0 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b9c3d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40a8fc0 con 0x7f80b406da40 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b9c3d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40a92a0 con 0x7f80b4142de0 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b41432b0 0x7f80b40a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b41432b0 0x7f80b40a8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57174/0 (socket says 172.21.15.72:57174) 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b3fff640 1 -- 172.21.15.72:0/1128799401 learned_addr learned my addr 172.21.15.72:0/1128799401 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b37fe640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b406da40 0x7f80b4159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b2ffd640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b3fff640 1 -- 172.21.15.72:0/1128799401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 msgr2=0x7f80b40a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b3fff640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b3fff640 1 -- 172.21.15.72:0/1128799401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b406da40 msgr2=0x7f80b4159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b3fff640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b406da40 0x7f80b4159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.913+0000 7f80b3fff640 1 -- 172.21.15.72:0/1128799401 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4009400 con 0x7f80b41432b0 2023-12-12T09:54:53.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.914+0000 7f80b3fff640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b41432b0 0x7f80b40a8990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c730 tx=0x7f80ac00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.914+0000 7f809bfff640 1 -- 172.21.15.72:0/1128799401 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac004d60 con 0x7f80b41432b0 2023-12-12T09:54:53.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.914+0000 7f809bfff640 1 -- 172.21.15.72:0/1128799401 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac00a640 con 0x7f80b41432b0 2023-12-12T09:54:53.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.914+0000 7f809bfff640 1 -- 172.21.15.72:0/1128799401 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00a9f0 con 0x7f80b41432b0 2023-12-12T09:54:53.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.914+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a80094d0 con 0x7f80b41432b0 2023-12-12T09:54:53.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.914+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b40a9860 con 0x7f80b41432b0 2023-12-12T09:54:53.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.915+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b40745d0 con 0x7f80b41432b0 2023-12-12T09:54:53.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.916+0000 7f809bfff640 1 -- 172.21.15.72:0/1128799401 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80ac00ab50 con 0x7f80b41432b0 2023-12-12T09:54:53.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.917+0000 7f809bfff640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f80a006ae40 0x7f80a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.917+0000 7f809bfff640 1 -- 172.21.15.72:0/1128799401 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f80ac088a70 con 0x7f80b41432b0 2023-12-12T09:54:53.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.917+0000 7f80b37fe640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f80a006ae40 0x7f80a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.918+0000 7f80b37fe640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f80a006ae40 0x7f80a006d300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009c60 tx=0x7f80a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:53.921+0000 7f809bfff640 1 -- 172.21.15.72:0/1128799401 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80ac057470 con 0x7f80b41432b0 2023-12-12T09:54:54.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.095+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80b40cc650 con 0x7f80b41432b0 2023-12-12T09:54:54.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.096+0000 7f809bfff640 1 -- 172.21.15.72:0/1128799401 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80ac057000 con 0x7f80b41432b0 2023-12-12T09:54:54.099 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:54:54.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.099+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f80a006ae40 msgr2=0x7f80a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.099+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f80a006ae40 0x7f80a006d300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009c60 tx=0x7f80a4008040 comp rx=0 tx=0).stop 2023-12-12T09:54:54.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.099+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b41432b0 msgr2=0x7f80b40a8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.099+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b41432b0 0x7f80b40a8990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c730 tx=0x7f80ac00cb30 comp rx=0 tx=0).stop 2023-12-12T09:54:54.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.100+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 shutdown_connections 2023-12-12T09:54:54.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.100+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f80a006ae40 0x7f80a006d300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.100+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b41432b0 0x7f80b40a8990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.100+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b4142de0 0x7f80b40a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.100+0000 7f80b9c3d640 1 --2- 172.21.15.72:0/1128799401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b406da40 0x7f80b4159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.103 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.100+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 >> 172.21.15.72:0/1128799401 conn(0x7f80b40cb850 msgr2=0x7f80b40cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:54.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.100+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 shutdown_connections 2023-12-12T09:54:54.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.100+0000 7f80b9c3d640 1 -- 172.21.15.72:0/1128799401 wait complete. 2023-12-12T09:54:54.123 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:54:54.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.324+0000 7f8d61e4f640 1 Processor -- start 2023-12-12T09:54:54.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.324+0000 7f8d61e4f640 1 -- start start 2023-12-12T09:54:54.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d61e4f640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8d5c15d0e0 0x7f8d5c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d61e4f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c072210 0x7f8d5c161530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d61e4f640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c160dc0 con 0x7f8d5c161a70 2023-12-12T09:54:54.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d61e4f640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5c070280 con 0x7f8d5c15d0e0 2023-12-12T09:54:54.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d61e4f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c070560 con 0x7f8d5c072210 2023-12-12T09:54:54.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d5b7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8d5c15d0e0 0x7f8d5c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:54.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d5bfff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f8d5c161a70 0x7f8d5c0c92e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:38742/0 (socket says 172.21.15.72:38742) 2023-12-12T09:54:54.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d5b7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8d5c15d0e0 0x7f8d5c0ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57176/0 (socket says 172.21.15.72:57176) 2023-12-12T09:54:54.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d5bfff640 1 -- 172.21.15.72:0/3156044245 learned_addr learned my addr 172.21.15.72:0/3156044245 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:54.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d5b7fe640 1 -- 172.21.15.72:0/3156044245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c072210 msgr2=0x7f8d5c161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:54.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d5b7fe640 1 --2- 172.21.15.72:0/3156044245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c072210 0x7f8d5c161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.326+0000 7f8d5b7fe640 1 -- 172.21.15.72:0/3156044245 >> v1:172.21.15.72:6789/0 conn(0x7f8d5c161a70 legacy=0x7f8d5c0c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.327+0000 7f8d5b7fe640 1 -- 172.21.15.72:0/3156044245 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c070840 con 0x7f8d5c15d0e0 2023-12-12T09:54:54.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.327+0000 7f8d5affd640 1 -- 172.21.15.72:0/3156044245 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207348012 0 0) 0x7f8d5c160dc0 con 0x7f8d5c161a70 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.328+0000 7f8d5b7fe640 1 --2- 172.21.15.72:0/3156044245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c15d0e0 0x7f8d5c0ce3a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009d90 tx=0x7f8d4c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa9012b62766adc server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.328+0000 7f8d5affd640 1 -- 172.21.15.72:0/3156044245 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c02cc50 con 0x7f8d5c15d0e0 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.328+0000 7f8d5affd640 1 -- 172.21.15.72:0/3156044245 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c02cdb0 con 0x7f8d5c15d0e0 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.328+0000 7f8d5affd640 1 -- 172.21.15.72:0/3156044245 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c034660 con 0x7f8d5c15d0e0 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.328+0000 7f8d61e4f640 1 -- 172.21.15.72:0/3156044245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c15d0e0 msgr2=0x7f8d5c0ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.328+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/3156044245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c15d0e0 0x7f8d5c0ce3a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009d90 tx=0x7f8d4c02bea0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.329+0000 7f8d61e4f640 1 -- 172.21.15.72:0/3156044245 shutdown_connections 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.329+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/3156044245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c072210 0x7f8d5c161530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.329+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/3156044245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c15d0e0 0x7f8d5c0ce3a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.329+0000 7f8d61e4f640 1 -- 172.21.15.72:0/3156044245 >> 172.21.15.72:0/3156044245 conn(0x7f8d5c0cee70 msgr2=0x7f8d5c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:54.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.329+0000 7f8d61e4f640 1 -- 172.21.15.72:0/3156044245 shutdown_connections 2023-12-12T09:54:54.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.329+0000 7f8d61e4f640 1 -- 172.21.15.72:0/3156044245 wait complete. 2023-12-12T09:54:54.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.330+0000 7f8d61e4f640 1 Processor -- start 2023-12-12T09:54:54.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.330+0000 7f8d61e4f640 1 -- start start 2023-12-12T09:54:54.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.331+0000 7f8d61e4f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c072210 0x7f8d5c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.331+0000 7f8d61e4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c15d0e0 0x7f8d5c0ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.331+0000 7f8d61e4f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c15ae00 0x7f8d5c0c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.331+0000 7f8d61e4f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0aca10 con 0x7f8d5c072210 2023-12-12T09:54:54.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.331+0000 7f8d61e4f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0accf0 con 0x7f8d5c15d0e0 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.331+0000 7f8d61e4f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0acfd0 con 0x7f8d5c15ae00 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c072210 0x7f8d5c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c072210 0x7f8d5c155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57190/0 (socket says 172.21.15.72:57190) 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 -- 172.21.15.72:0/56974623 learned_addr learned my addr 172.21.15.72:0/56974623 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d52ffd640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c15d0e0 0x7f8d5c0ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 -- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c15ae00 msgr2=0x7f8d5c0c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5bfff640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c15ae00 0x7f8d5c0c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c15ae00 0x7f8d5c0c3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 -- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c15d0e0 msgr2=0x7f8d5c0ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c15d0e0 0x7f8d5c0ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 -- 172.21.15.72:0/56974623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c009a70 con 0x7f8d5c072210 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d52ffd640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c15d0e0 0x7f8d5c0ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.332+0000 7f8d5b7fe640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c072210 0x7f8d5c155220 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009d90 tx=0x7f8d4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.333+0000 7f8d58ff9640 1 -- 172.21.15.72:0/56974623 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c03e610 con 0x7f8d5c072210 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.333+0000 7f8d58ff9640 1 -- 172.21.15.72:0/56974623 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c03e770 con 0x7f8d5c072210 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.333+0000 7f8d58ff9640 1 -- 172.21.15.72:0/56974623 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c03eb60 con 0x7f8d5c072210 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.333+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c0ad2b0 con 0x7f8d5c072210 2023-12-12T09:54:54.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.333+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c0ac720 con 0x7f8d5c072210 2023-12-12T09:54:54.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.334+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c074450 con 0x7f8d5c072210 2023-12-12T09:54:54.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.335+0000 7f8d58ff9640 1 -- 172.21.15.72:0/56974623 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d4c03b050 con 0x7f8d5c072210 2023-12-12T09:54:54.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.335+0000 7f8d58ff9640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8d3c06ae70 0x7f8d3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.335+0000 7f8d58ff9640 1 -- 172.21.15.72:0/56974623 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8d4c0abe80 con 0x7f8d5c072210 2023-12-12T09:54:54.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.336+0000 7f8d52ffd640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8d3c06ae70 0x7f8d3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:54.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.336+0000 7f8d52ffd640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8d3c06ae70 0x7f8d3c06d330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c1e6f70 tx=0x7f8d44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:54.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.340+0000 7f8d58ff9640 1 -- 172.21.15.72:0/56974623 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d4c076d60 con 0x7f8d5c072210 2023-12-12T09:54:54.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.529+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d5c0ae620 con 0x7f8d5c072210 2023-12-12T09:54:54.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.530+0000 7f8d58ff9640 1 -- 172.21.15.72:0/56974623 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d4c07a380 con 0x7f8d5c072210 2023-12-12T09:54:54.532 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:54:54.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8d3c06ae70 msgr2=0x7f8d3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8d3c06ae70 0x7f8d3c06d330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c1e6f70 tx=0x7f8d44008040 comp rx=0 tx=0).stop 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c072210 msgr2=0x7f8d5c155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c072210 0x7f8d5c155220 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009d90 tx=0x7f8d4c004a40 comp rx=0 tx=0).stop 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 shutdown_connections 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8d3c06ae70 0x7f8d3c06d330 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d5c15ae00 0x7f8d5c0c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d5c15d0e0 0x7f8d5c0ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 --2- 172.21.15.72:0/56974623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d5c072210 0x7f8d5c155220 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.532+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 >> 172.21.15.72:0/56974623 conn(0x7f8d5c0cee70 msgr2=0x7f8d5c0af330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.533+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 shutdown_connections 2023-12-12T09:54:54.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.533+0000 7f8d61e4f640 1 -- 172.21.15.72:0/56974623 wait complete. 2023-12-12T09:54:54.552 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:54:54.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.757+0000 7f370c61e640 1 Processor -- start 2023-12-12T09:54:54.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.757+0000 7f370c61e640 1 -- start start 2023-12-12T09:54:54.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370c61e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3704157a60 0x7f370406c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370c61e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370406de20 0x7f370415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370c61e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370406ca40 con 0x7f370415e0e0 2023-12-12T09:54:54.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370c61e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370415e550 con 0x7f3704157a60 2023-12-12T09:54:54.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370c61e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3704071cd0 con 0x7f370406de20 2023-12-12T09:54:54.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370a3ba640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3704157a60 0x7f370406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:54.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370abbb640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f370415e0e0 0x7f370415a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:38758/0 (socket says 172.21.15.72:38758) 2023-12-12T09:54:54.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370a3ba640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3704157a60 0x7f370406c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57206/0 (socket says 172.21.15.72:57206) 2023-12-12T09:54:54.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.759+0000 7f370abbb640 1 -- 172.21.15.72:0/1306827499 learned_addr learned my addr 172.21.15.72:0/1306827499 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:54.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.760+0000 7f370a3ba640 1 -- 172.21.15.72:0/1306827499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370406de20 msgr2=0x7f370415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:54.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.760+0000 7f370a3ba640 1 --2- 172.21.15.72:0/1306827499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370406de20 0x7f370415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.760+0000 7f370a3ba640 1 -- 172.21.15.72:0/1306827499 >> v1:172.21.15.72:6789/0 conn(0x7f370415e0e0 legacy=0x7f370415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.760+0000 7f370a3ba640 1 -- 172.21.15.72:0/1306827499 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704070f10 con 0x7f3704157a60 2023-12-12T09:54:54.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.761+0000 7f370a3ba640 1 --2- 172.21.15.72:0/1306827499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3704157a60 0x7f370406c410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f37000097a0 tx=0x7f370002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9803dc396ad17c98 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:54.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.761+0000 7f37093b8640 1 -- 172.21.15.72:0/1306827499 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700031780 con 0x7f3704157a60 2023-12-12T09:54:54.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.761+0000 7f37093b8640 1 -- 172.21.15.72:0/1306827499 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37000318e0 con 0x7f3704157a60 2023-12-12T09:54:54.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.761+0000 7f37093b8640 1 -- 172.21.15.72:0/1306827499 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700031c70 con 0x7f3704157a60 2023-12-12T09:54:54.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.761+0000 7f370c61e640 1 -- 172.21.15.72:0/1306827499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3704157a60 msgr2=0x7f370406c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.761+0000 7f370c61e640 1 --2- 172.21.15.72:0/1306827499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3704157a60 0x7f370406c410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f37000097a0 tx=0x7f370002b820 comp rx=0 tx=0).stop 2023-12-12T09:54:54.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.762+0000 7f370c61e640 1 -- 172.21.15.72:0/1306827499 shutdown_connections 2023-12-12T09:54:54.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.762+0000 7f370c61e640 1 --2- 172.21.15.72:0/1306827499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370406de20 0x7f370415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.762+0000 7f370c61e640 1 --2- 172.21.15.72:0/1306827499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3704157a60 0x7f370406c410 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.762+0000 7f370c61e640 1 -- 172.21.15.72:0/1306827499 >> 172.21.15.72:0/1306827499 conn(0x7f37040b0a90 msgr2=0x7f37040b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:54.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.762+0000 7f370c61e640 1 -- 172.21.15.72:0/1306827499 shutdown_connections 2023-12-12T09:54:54.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.762+0000 7f370c61e640 1 -- 172.21.15.72:0/1306827499 wait complete. 2023-12-12T09:54:54.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.763+0000 7f370c61e640 1 Processor -- start 2023-12-12T09:54:54.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.764+0000 7f370c61e640 1 -- start start 2023-12-12T09:54:54.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.764+0000 7f370c61e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370406de20 0x7f37040add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370c61e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3704157a60 0x7f370413f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370c61e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370415e0e0 0x7f3704155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370c61e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3704142840 con 0x7f370415e0e0 2023-12-12T09:54:54.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370c61e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040ae2c0 con 0x7f3704157a60 2023-12-12T09:54:54.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370c61e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3704141c30 con 0x7f370406de20 2023-12-12T09:54:54.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370abbb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370415e0e0 0x7f3704155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:54.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370abbb640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370415e0e0 0x7f3704155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57214/0 (socket says 172.21.15.72:57214) 2023-12-12T09:54:54.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370abbb640 1 -- 172.21.15.72:0/2509138149 learned_addr learned my addr 172.21.15.72:0/2509138149 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:54.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370abbb640 1 -- 172.21.15.72:0/2509138149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370406de20 msgr2=0x7f37040add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370abbb640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370406de20 0x7f37040add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370abbb640 1 -- 172.21.15.72:0/2509138149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3704157a60 msgr2=0x7f370413f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370abbb640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3704157a60 0x7f370413f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.765+0000 7f370abbb640 1 -- 172.21.15.72:0/2509138149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3700009400 con 0x7f370415e0e0 2023-12-12T09:54:54.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.766+0000 7f370abbb640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370415e0e0 0x7f3704155090 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00ca80 tx=0x7f36fc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:54.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.766+0000 7f36faffd640 1 -- 172.21.15.72:0/2509138149 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc00ac30 con 0x7f370415e0e0 2023-12-12T09:54:54.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.766+0000 7f36faffd640 1 -- 172.21.15.72:0/2509138149 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc00ad90 con 0x7f370415e0e0 2023-12-12T09:54:54.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.766+0000 7f36faffd640 1 -- 172.21.15.72:0/2509138149 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc012600 con 0x7f370415e0e0 2023-12-12T09:54:54.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.766+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3704141f10 con 0x7f370415e0e0 2023-12-12T09:54:54.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.766+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37041424d0 con 0x7f370415e0e0 2023-12-12T09:54:54.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.767+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37040745d0 con 0x7f370415e0e0 2023-12-12T09:54:54.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.768+0000 7f36faffd640 1 -- 172.21.15.72:0/2509138149 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36fc007070 con 0x7f370415e0e0 2023-12-12T09:54:54.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.770+0000 7f36faffd640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f36e806ae40 0x7f36e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:54.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.770+0000 7f36faffd640 1 -- 172.21.15.72:0/2509138149 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f36fc089570 con 0x7f370415e0e0 2023-12-12T09:54:54.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.770+0000 7f370a3ba640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f36e806ae40 0x7f36e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:54.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.771+0000 7f370a3ba640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f36e806ae40 0x7f36e806d300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3700036000 tx=0x7f3700030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:54.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.774+0000 7f36faffd640 1 -- 172.21.15.72:0/2509138149 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36fc0544f0 con 0x7f370415e0e0 2023-12-12T09:54:54.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.979+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3704142980 con 0x7f370415e0e0 2023-12-12T09:54:54.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.980+0000 7f36faffd640 1 -- 172.21.15.72:0/2509138149 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f36fc057b10 con 0x7f370415e0e0 2023-12-12T09:54:54.982 INFO:teuthology.orchestra.run.smithi072.stderr:listed 0 entries 2023-12-12T09:54:54.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f36e806ae40 msgr2=0x7f36e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f36e806ae40 0x7f36e806d300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3700036000 tx=0x7f3700030000 comp rx=0 tx=0).stop 2023-12-12T09:54:54.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370415e0e0 msgr2=0x7f3704155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:54.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370415e0e0 0x7f3704155090 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00ca80 tx=0x7f36fc00cf40 comp rx=0 tx=0).stop 2023-12-12T09:54:54.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 shutdown_connections 2023-12-12T09:54:54.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f36e806ae40 0x7f36e806d300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370415e0e0 0x7f3704155090 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3704157a60 0x7f370413f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.982+0000 7f370c61e640 1 --2- 172.21.15.72:0/2509138149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f370406de20 0x7f37040add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:54.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.983+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 >> 172.21.15.72:0/2509138149 conn(0x7f37040b0a90 msgr2=0x7f3704157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:54.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.983+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 shutdown_connections 2023-12-12T09:54:54.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:54.983+0000 7f370c61e640 1 -- 172.21.15.72:0/2509138149 wait complete. 2023-12-12T09:54:55.003 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-12T09:54:55.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:55.004 DEBUG:teuthology.orchestra.run.smithi072:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-12T09:54:55.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:54:55.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:54:55.188 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-12T09:54:55.227 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:54:55.227 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:55.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054290,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-12T09:54:55.268 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:54:55.268 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:55.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054258,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-12T09:54:55.311 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:54:55.311 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:54:55.362 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054290,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-12T09:54:55.362 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.573+0000 7fbb3de87640 1 Processor -- start 2023-12-12T09:54:55.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.573+0000 7fbb3de87640 1 -- start start 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.575+0000 7fbb3de87640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb3815d580 0x7fbb380adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.575+0000 7fbb3de87640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3806c690 0x7fbb380d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.575+0000 7fbb3de87640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb380b07f0 con 0x7fbb380cfc50 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.575+0000 7fbb3de87640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb380d0620 con 0x7fbb3815d580 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.575+0000 7fbb3de87640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3815d960 con 0x7fbb3806c690 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.576+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3806c690 0x7fbb380d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.576+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3806c690 0x7fbb380d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:53644/0 (socket says 172.21.15.72:53644) 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.576+0000 7fbb37fff640 1 -- 172.21.15.72:0/3637383778 learned_addr learned my addr 172.21.15.72:0/3637383778 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.576+0000 7fbb377fe640 1 --2- 172.21.15.72:0/3637383778 >> v2:172.21.15.113:3300/0 conn(0x7fbb3815d580 0x7fbb380adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:55.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.576+0000 7fbb37fff640 1 -- 172.21.15.72:0/3637383778 >> v2:172.21.15.113:3300/0 conn(0x7fbb3815d580 msgr2=0x7fbb380adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:55.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.576+0000 7fbb37fff640 1 --2- 172.21.15.72:0/3637383778 >> v2:172.21.15.113:3300/0 conn(0x7fbb3815d580 0x7fbb380adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.576+0000 7fbb37fff640 1 -- 172.21.15.72:0/3637383778 >> v1:172.21.15.113:6789/0 conn(0x7fbb380cfc50 legacy=0x7fbb3806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:55.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.576+0000 7fbb37fff640 1 -- 172.21.15.72:0/3637383778 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380b0a40 con 0x7fbb3806c690 2023-12-12T09:54:55.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.578+0000 7fbb37fff640 1 --2- 172.21.15.72:0/3637383778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3806c690 0x7fbb380d00e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009870 tx=0x7fbb3002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61c559ba4032008 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:55.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.578+0000 7fbb367fc640 1 -- 172.21.15.72:0/3637383778 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30030840 con 0x7fbb3806c690 2023-12-12T09:54:55.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.578+0000 7fbb367fc640 1 -- 172.21.15.72:0/3637383778 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb300309a0 con 0x7fbb3806c690 2023-12-12T09:54:55.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.578+0000 7fbb367fc640 1 -- 172.21.15.72:0/3637383778 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30030d30 con 0x7fbb3806c690 2023-12-12T09:54:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.579+0000 7fbb3de87640 1 -- 172.21.15.72:0/3637383778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3806c690 msgr2=0x7fbb380d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.579+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3637383778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3806c690 0x7fbb380d00e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009870 tx=0x7fbb3002b980 comp rx=0 tx=0).stop 2023-12-12T09:54:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.579+0000 7fbb3de87640 1 -- 172.21.15.72:0/3637383778 shutdown_connections 2023-12-12T09:54:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.579+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3637383778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3806c690 0x7fbb380d00e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.579+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3637383778 >> v2:172.21.15.113:3300/0 conn(0x7fbb3815d580 0x7fbb380adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.579+0000 7fbb3de87640 1 -- 172.21.15.72:0/3637383778 >> 172.21.15.72:0/3637383778 conn(0x7fbb380d0e90 msgr2=0x7fbb380d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.579+0000 7fbb3de87640 1 -- 172.21.15.72:0/3637383778 shutdown_connections 2023-12-12T09:54:55.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.579+0000 7fbb3de87640 1 -- 172.21.15.72:0/3637383778 wait complete. 2023-12-12T09:54:55.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.581+0000 7fbb3de87640 1 Processor -- start 2023-12-12T09:54:55.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.581+0000 7fbb3de87640 1 -- start start 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.582+0000 7fbb3de87640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb3806c690 0x7fbb38157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.582+0000 7fbb3de87640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb380cfc50 0x7fbb381602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.582+0000 7fbb3de87640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3815d580 0x7fbb380b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.582+0000 7fbb3de87640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb38154e60 con 0x7fbb380cfc50 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.582+0000 7fbb3de87640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380c65d0 con 0x7fbb3806c690 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.582+0000 7fbb3de87640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380b6750 con 0x7fbb3815d580 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb36ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb380cfc50 0x7fbb381602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb36ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb380cfc50 0x7fbb381602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53140/0 (socket says 172.21.15.72:53140) 2023-12-12T09:54:55.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb36ffd640 1 -- 172.21.15.72:0/3173625714 learned_addr learned my addr 172.21.15.72:0/3173625714 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:55.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb37fff640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3815d580 0x7fbb380b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:55.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb377fe640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb3806c690 0x7fbb38157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb36ffd640 1 -- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3815d580 msgr2=0x7fbb380b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb36ffd640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3815d580 0x7fbb380b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb36ffd640 1 -- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb3806c690 msgr2=0x7fbb38157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb36ffd640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb3806c690 0x7fbb38157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb36ffd640 1 -- 172.21.15.72:0/3173625714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb300094d0 con 0x7fbb380cfc50 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.583+0000 7fbb377fe640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb3806c690 0x7fbb38157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.584+0000 7fbb36ffd640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb380cfc50 0x7fbb381602e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800c8a0 tx=0x7fbb2800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.584+0000 7fbb2ffff640 1 -- 172.21.15.72:0/3173625714 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28003b70 con 0x7fbb380cfc50 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.584+0000 7fbb2ffff640 1 -- 172.21.15.72:0/3173625714 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb28003cf0 con 0x7fbb380cfc50 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.584+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb380b4bb0 con 0x7fbb380cfc50 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.584+0000 7fbb2ffff640 1 -- 172.21.15.72:0/3173625714 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800aa90 con 0x7fbb380cfc50 2023-12-12T09:54:55.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.584+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb380c6820 con 0x7fbb380cfc50 2023-12-12T09:54:55.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.585+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb380745d0 con 0x7fbb380cfc50 2023-12-12T09:54:55.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.586+0000 7fbb2ffff640 1 -- 172.21.15.72:0/3173625714 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb2800abf0 con 0x7fbb380cfc50 2023-12-12T09:54:55.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.587+0000 7fbb2ffff640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbb1806ae40 0x7fbb1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:55.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.587+0000 7fbb2ffff640 1 -- 172.21.15.72:0/3173625714 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fbb280895b0 con 0x7fbb380cfc50 2023-12-12T09:54:55.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.588+0000 7fbb377fe640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbb1806ae40 0x7fbb1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:55.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.588+0000 7fbb377fe640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbb1806ae40 0x7fbb1806d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009a20 tx=0x7fbb20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:55.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.592+0000 7fbb2ffff640 1 -- 172.21.15.72:0/3173625714 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb28054570 con 0x7fbb380cfc50 2023-12-12T09:54:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.819+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb3806b9f0 con 0x7fbb380cfc50 2023-12-12T09:54:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.820+0000 7fbb2ffff640 1 -- 172.21.15.72:0/3173625714 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fbb28003e60 con 0x7fbb380cfc50 2023-12-12T09:54:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:55.822 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/130800926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":130800926},{"type":"v1","addr":"172.21.15.72:6836","nonce":130800926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/318582100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":318582100},{"type":"v1","addr":"172.21.15.72:6837","nonce":318582100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2380516950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2380516950},{"type":"v1","addr":"172.21.15.113:6835","nonce":2380516950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:54:32.882443+0000","modified":"2023-12-12T09:54:35.940697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/4181915308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4181915308},{"type":"v1","addr":"172.21.15.113:6834","nonce":4181915308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-12T09:54:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 25 2023-12-12T09:54:55.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.822+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbb1806ae40 msgr2=0x7fbb1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.822+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbb1806ae40 0x7fbb1806d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009a20 tx=0x7fbb20008040 comp rx=0 tx=0).stop 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.822+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb380cfc50 msgr2=0x7fbb381602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.822+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb380cfc50 0x7fbb381602e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800c8a0 tx=0x7fbb2800cd60 comp rx=0 tx=0).stop 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.822+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 shutdown_connections 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.823+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fbb1806ae40 0x7fbb1806d300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.823+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb3815d580 0x7fbb380b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.823+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb380cfc50 0x7fbb381602e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.823+0000 7fbb3de87640 1 --2- 172.21.15.72:0/3173625714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb3806c690 0x7fbb38157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.823+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 >> 172.21.15.72:0/3173625714 conn(0x7fbb380d0e90 msgr2=0x7fbb3806bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.823+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 shutdown_connections 2023-12-12T09:54:55.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:55.823+0000 7fbb3de87640 1 -- 172.21.15.72:0/3173625714 wait complete. 2023-12-12T09:54:55.842 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:56.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.046+0000 7ff5d6fb6640 1 Processor -- start 2023-12-12T09:54:56.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.046+0000 7ff5d6fb6640 1 -- start start 2023-12-12T09:54:56.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.047+0000 7ff5d6fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 0x7ff5d00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.047+0000 7ff5d6fb6640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff5d0072180 0x7ff5d00b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5d6fb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d00b3bb0 con 0x7ff5d006c410 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5d6fb6640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d01441e0 con 0x7ff5d0072180 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5d6fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d0072680 con 0x7ff5d0143e00 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5c7fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff5d0072180 0x7ff5d00b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5c7fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff5d0072180 0x7ff5d00b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53142/0 (socket says 172.21.15.72:53142) 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5c7fff640 1 -- 172.21.15.72:0/1542678267 learned_addr learned my addr 172.21.15.72:0/1542678267 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5d4d52640 1 --2- 172.21.15.72:0/1542678267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 0x7ff5d00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5c7fff640 1 -- 172.21.15.72:0/1542678267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 msgr2=0x7ff5d00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5c7fff640 1 --2- 172.21.15.72:0/1542678267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 0x7ff5d00b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5c7fff640 1 -- 172.21.15.72:0/1542678267 >> v1:172.21.15.113:6789/0 conn(0x7ff5d006c410 legacy=0x7ff5d0070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:56.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.048+0000 7ff5c7fff640 1 -- 172.21.15.72:0/1542678267 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d00b1e30 con 0x7ff5d0072180 2023-12-12T09:54:56.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.050+0000 7ff5c7fff640 1 --2- 172.21.15.72:0/1542678267 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d00b4580 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009e50 tx=0x7ff5b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892bc04f8a109dba server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.050+0000 7ff5cffff640 1 -- 172.21.15.72:0/1542678267 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b802ca70 con 0x7ff5d0072180 2023-12-12T09:54:56.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.050+0000 7ff5cffff640 1 -- 172.21.15.72:0/1542678267 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b802cbd0 con 0x7ff5d0072180 2023-12-12T09:54:56.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.050+0000 7ff5cffff640 1 -- 172.21.15.72:0/1542678267 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80343f0 con 0x7ff5d0072180 2023-12-12T09:54:56.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.051+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/1542678267 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 msgr2=0x7ff5d00b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.051+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/1542678267 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d00b4580 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009e50 tx=0x7ff5b802bed0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.051+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/1542678267 shutdown_connections 2023-12-12T09:54:56.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.051+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/1542678267 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d00b4580 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.051+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/1542678267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 0x7ff5d00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.051+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/1542678267 >> 172.21.15.72:0/1542678267 conn(0x7ff5d00b0a90 msgr2=0x7ff5d00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:56.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.051+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/1542678267 shutdown_connections 2023-12-12T09:54:56.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.051+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/1542678267 wait complete. 2023-12-12T09:54:56.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.053+0000 7ff5d6fb6640 1 Processor -- start 2023-12-12T09:54:56.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.053+0000 7ff5d6fb6640 1 -- start start 2023-12-12T09:54:56.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.054+0000 7ff5d6fb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d006c410 0x7ff5d0136a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.054+0000 7ff5d6fb6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d013a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.054+0000 7ff5d6fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 0x7ff5d00adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.054+0000 7ff5d6fb6640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d013b0d0 con 0x7ff5d0072180 2023-12-12T09:54:56.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.054+0000 7ff5d6fb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d013b3b0 con 0x7ff5d006c410 2023-12-12T09:54:56.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.054+0000 7ff5d6fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d013b690 con 0x7ff5d0143e00 2023-12-12T09:54:56.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d013a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d013a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53156/0 (socket says 172.21.15.72:53156) 2023-12-12T09:54:56.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 -- 172.21.15.72:0/958161179 learned_addr learned my addr 172.21.15.72:0/958161179 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:56.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 -- 172.21.15.72:0/958161179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 msgr2=0x7ff5d00adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 0x7ff5d00adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 -- 172.21.15.72:0/958161179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d006c410 msgr2=0x7ff5d0136a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d006c410 0x7ff5d0136a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 -- 172.21.15.72:0/958161179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8009b30 con 0x7ff5d0072180 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.055+0000 7ff5c7fff640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d013a610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000f80 tx=0x7ff5b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.056+0000 7ff5cdffb640 1 -- 172.21.15.72:0/958161179 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b803e450 con 0x7ff5d0072180 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.056+0000 7ff5cdffb640 1 -- 172.21.15.72:0/958161179 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b803e5b0 con 0x7ff5d0072180 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.056+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d013b970 con 0x7ff5d0072180 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.056+0000 7ff5cdffb640 1 -- 172.21.15.72:0/958161179 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b803e960 con 0x7ff5d0072180 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.056+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d0141540 con 0x7ff5d0072180 2023-12-12T09:54:56.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.057+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d013be90 con 0x7ff5d0072180 2023-12-12T09:54:56.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.058+0000 7ff5cdffb640 1 -- 172.21.15.72:0/958161179 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5b803eac0 con 0x7ff5d0072180 2023-12-12T09:54:56.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.059+0000 7ff5cdffb640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5b006ae40 0x7ff5b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.059+0000 7ff5cdffb640 1 -- 172.21.15.72:0/958161179 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6087+0+0 (secure 0 0 0) 0x7ff5b80ac680 con 0x7ff5d0072180 2023-12-12T09:54:56.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.059+0000 7ff5d4d52640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5b006ae40 0x7ff5b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.060+0000 7ff5d4d52640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5b006ae40 0x7ff5b006d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0006fd0 tx=0x7ff5c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.064+0000 7ff5cdffb640 1 -- 172.21.15.72:0/958161179 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5d013be90 con 0x7ff5d0072180 2023-12-12T09:54:56.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.255+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5d013be90 con 0x7ff5d0072180 2023-12-12T09:54:56.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.258+0000 7ff5cdffb640 1 -- 172.21.15.72:0/958161179 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16812 (secure 0 0 0) 0x7ff5d013be90 con 0x7ff5d0072180 2023-12-12T09:54:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:56.260 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":52,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:55.459908+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:54:30.694215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-12T09:54:31.894291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.c","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:56.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.260+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5b006ae40 msgr2=0x7ff5b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.260+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5b006ae40 0x7ff5b006d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0006fd0 tx=0x7ff5c0008040 comp rx=0 tx=0).stop 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.260+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 msgr2=0x7ff5d013a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.260+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d013a610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000f80 tx=0x7ff5b8004a40 comp rx=0 tx=0).stop 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.261+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 shutdown_connections 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.261+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff5b006ae40 0x7ff5b006d300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.261+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5d0143e00 0x7ff5d00adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.261+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5d0072180 0x7ff5d013a610 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.261+0000 7ff5d6fb6640 1 --2- 172.21.15.72:0/958161179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5d006c410 0x7ff5d0136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.261+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 >> 172.21.15.72:0/958161179 conn(0x7ff5d00b0a90 msgr2=0x7ff5d00af480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.261+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 shutdown_connections 2023-12-12T09:54:56.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.261+0000 7ff5d6fb6640 1 -- 172.21.15.72:0/958161179 wait complete. 2023-12-12T09:54:56.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:01.617276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:54:30.694215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:54:31.894291+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:54:56.282 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:54:56.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.499+0000 7fcc585d5640 1 Processor -- start 2023-12-12T09:54:56.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.499+0000 7fcc585d5640 1 -- start start 2023-12-12T09:54:56.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.500+0000 7fcc585d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc5015af50 0x7fcc5006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.500+0000 7fcc585d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcc5006a460 0x7fcc5006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.500+0000 7fcc585d5640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc50071cd0 con 0x7fcc500ce7e0 2023-12-12T09:54:56.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.500+0000 7fcc585d5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5006ae30 con 0x7fcc5006a460 2023-12-12T09:54:56.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.500+0000 7fcc585d5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500717e0 con 0x7fcc5015af50 2023-12-12T09:54:56.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc55b70640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fcc500ce7e0 0x7fcc5015ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36944/0 (socket says 172.21.15.72:36944) 2023-12-12T09:54:56.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc55b70640 1 -- 172.21.15.72:0/2505806997 learned_addr learned my addr 172.21.15.72:0/2505806997 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc56b72640 1 --2- 172.21.15.72:0/2505806997 >> v2:172.21.15.113:3300/0 conn(0x7fcc5006a460 0x7fcc5006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc56371640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc5015af50 0x7fcc5006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043701392 0 0) 0x7fcc50071cd0 con 0x7fcc500ce7e0 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc3c003520 con 0x7fcc500ce7e0 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2386070875 0 0) 0x7fcc3c003520 con 0x7fcc500ce7e0 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc5015af50 msgr2=0x7fcc5006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc5015af50 0x7fcc5006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 >> v2:172.21.15.113:3300/0 conn(0x7fcc5006a460 msgr2=0x7fcc5006a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 --2- 172.21.15.72:0/2505806997 >> v2:172.21.15.113:3300/0 conn(0x7fcc5006a460 0x7fcc5006a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5006c6b0 con 0x7fcc500ce7e0 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.501+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fcc40001670 con 0x7fcc500ce7e0 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 >> v1:172.21.15.72:6789/0 conn(0x7fcc500ce7e0 legacy=0x7fcc5015ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc5536f640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3c003970 0x7fcc3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc5536f640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc3c0047c0 0x7fcc3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc5536f640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc5015af50 0x7fcc3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5006ae30 con 0x7fcc3c0047c0 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500717e0 con 0x7fcc3c003970 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c00bab0 con 0x7fcc5015af50 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc55b70640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc3c0047c0 0x7fcc3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc56371640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3c003970 0x7fcc3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc55b70640 1 -- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc5015af50 msgr2=0x7fcc3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc55b70640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc5015af50 0x7fcc3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc55b70640 1 -- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3c003970 msgr2=0x7fcc3c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc55b70640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3c003970 0x7fcc3c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc55b70640 1 -- 172.21.15.72:0/2505806997 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c003520 con 0x7fcc3c0047c0 2023-12-12T09:54:56.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.502+0000 7fcc56371640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc3c003970 0x7fcc3c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:56.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.503+0000 7fcc55b70640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc3c0047c0 0x7fcc3c004c40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000c240 tx=0x7fcc3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbef55906bfb0ef2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.504+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4002cb00 con 0x7fcc3c0047c0 2023-12-12T09:54:56.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.504+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4002cc60 con 0x7fcc3c0047c0 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.504+0000 7fcc5536f640 1 -- 172.21.15.72:0/2505806997 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40005af0 con 0x7fcc3c0047c0 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.504+0000 7fcc585d5640 1 -- 172.21.15.72:0/2505806997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc3c0047c0 msgr2=0x7fcc3c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.504+0000 7fcc585d5640 1 --2- 172.21.15.72:0/2505806997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc3c0047c0 0x7fcc3c004c40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000c240 tx=0x7fcc3c0057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.504+0000 7fcc56b72640 1 -- 172.21.15.72:0/2505806997 reap_dead start 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.504+0000 7fcc585d5640 1 -- 172.21.15.72:0/2505806997 shutdown_connections 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.504+0000 7fcc585d5640 1 -- 172.21.15.72:0/2505806997 >> 172.21.15.72:0/2505806997 conn(0x7fcc500cee70 msgr2=0x7fcc500cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.505+0000 7fcc585d5640 1 -- 172.21.15.72:0/2505806997 shutdown_connections 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.505+0000 7fcc585d5640 1 -- 172.21.15.72:0/2505806997 wait complete. 2023-12-12T09:54:56.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.506+0000 7fcc585d5640 1 Processor -- start 2023-12-12T09:54:56.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.507+0000 7fcc585d5640 1 -- start start 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.507+0000 7fcc585d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3c0047c0 0x7fcc500ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.507+0000 7fcc585d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc500abfc0 0x7fcc50155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.507+0000 7fcc585d5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc500700e0 0x7fcc50070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc585d5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50070ad0 con 0x7fcc500700e0 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc585d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5009cf20 con 0x7fcc500abfc0 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc585d5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500af0f0 con 0x7fcc3c0047c0 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc56b72640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc500700e0 0x7fcc50070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc56b72640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc500700e0 0x7fcc50070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53176/0 (socket says 172.21.15.72:53176) 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc56b72640 1 -- 172.21.15.72:0/3898848942 learned_addr learned my addr 172.21.15.72:0/3898848942 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:56.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc55b70640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc500abfc0 0x7fcc50155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc56b72640 1 -- 172.21.15.72:0/3898848942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3c0047c0 msgr2=0x7fcc500ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc56b72640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3c0047c0 0x7fcc500ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc56b72640 1 -- 172.21.15.72:0/3898848942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc500abfc0 msgr2=0x7fcc50155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc56b72640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc500abfc0 0x7fcc50155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.508+0000 7fcc56b72640 1 -- 172.21.15.72:0/3898848942 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50156060 con 0x7fcc500700e0 2023-12-12T09:54:56.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.509+0000 7fcc56b72640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc500700e0 0x7fcc50070590 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcc5006b640 tx=0x7fcc4800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.509+0000 7fcc46ffd640 1 -- 172.21.15.72:0/3898848942 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48010040 con 0x7fcc500700e0 2023-12-12T09:54:56.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.509+0000 7fcc46ffd640 1 -- 172.21.15.72:0/3898848942 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4800bae0 con 0x7fcc500700e0 2023-12-12T09:54:56.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.509+0000 7fcc46ffd640 1 -- 172.21.15.72:0/3898848942 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4800be70 con 0x7fcc500700e0 2023-12-12T09:54:56.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.509+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc500aeb30 con 0x7fcc500700e0 2023-12-12T09:54:56.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.509+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5009a3d0 con 0x7fcc500700e0 2023-12-12T09:54:56.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.510+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5009a230 con 0x7fcc500700e0 2023-12-12T09:54:56.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.511+0000 7fcc46ffd640 1 -- 172.21.15.72:0/3898848942 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc4801e410 con 0x7fcc500700e0 2023-12-12T09:54:56.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.512+0000 7fcc46ffd640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc3406ae70 0x7fcc3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.512+0000 7fcc46ffd640 1 -- 172.21.15.72:0/3898848942 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fcc48089a80 con 0x7fcc500700e0 2023-12-12T09:54:56.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.512+0000 7fcc56371640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc3406ae70 0x7fcc3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.513+0000 7fcc56371640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc3406ae70 0x7fcc3406d330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c006f00 tx=0x7fcc4c006e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.516+0000 7fcc46ffd640 1 -- 172.21.15.72:0/3898848942 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc5009a230 con 0x7fcc500700e0 2023-12-12T09:54:56.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.710+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc5009a230 con 0x7fcc500700e0 2023-12-12T09:54:56.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.711+0000 7fcc46ffd640 1 -- 172.21.15.72:0/3898848942 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc5009a230 con 0x7fcc500700e0 2023-12-12T09:54:56.714 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-12-12T09:54:56.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc3406ae70 msgr2=0x7fcc3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc3406ae70 0x7fcc3406d330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c006f00 tx=0x7fcc4c006e30 comp rx=0 tx=0).stop 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc500700e0 msgr2=0x7fcc50070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc500700e0 0x7fcc50070590 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcc5006b640 tx=0x7fcc4800b480 comp rx=0 tx=0).stop 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 shutdown_connections 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fcc3406ae70 0x7fcc3406d330 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc500700e0 0x7fcc50070590 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc500abfc0 0x7fcc50155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 --2- 172.21.15.72:0/3898848942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc3c0047c0 0x7fcc500ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 >> 172.21.15.72:0/3898848942 conn(0x7fcc500cee70 msgr2=0x7fcc50159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 shutdown_connections 2023-12-12T09:54:56.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.714+0000 7fcc585d5640 1 -- 172.21.15.72:0/3898848942 wait complete. 2023-12-12T09:54:56.734 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:54:56.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.943+0000 7f02c9a00640 1 Processor -- start 2023-12-12T09:54:56.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.943+0000 7f02c9a00640 1 -- start start 2023-12-12T09:54:56.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.945+0000 7f02c9a00640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f02c406db50 0x7f02c4141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.945+0000 7f02c9a00640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c415b780 con 0x7f02c406d770 2023-12-12T09:54:56.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.945+0000 7f02c9a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c40b2510 con 0x7f02c4141790 2023-12-12T09:54:56.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.945+0000 7f02c9a00640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c4141c20 con 0x7f02c406db50 2023-12-12T09:54:56.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.945+0000 7f02c27fc640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f02c406db50 0x7f02c4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.945+0000 7f02c2ffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f02c406d770 0x7f02c40b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36956/0 (socket says 172.21.15.72:36956) 2023-12-12T09:54:56.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.945+0000 7f02c2ffd640 1 -- 172.21.15.72:0/1132047453 learned_addr learned my addr 172.21.15.72:0/1132047453 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:56.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.946+0000 7f02c27fc640 1 -- 172.21.15.72:0/1132047453 >> v1:172.21.15.113:6789/0 conn(0x7f02c4141790 legacy=0x7f02c40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:56.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.946+0000 7f02c27fc640 1 -- 172.21.15.72:0/1132047453 >> v1:172.21.15.72:6789/0 conn(0x7f02c406d770 legacy=0x7f02c40b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.946+0000 7f02c1ffb640 1 -- 172.21.15.72:0/1132047453 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571724694 0 0) 0x7f02c415b780 con 0x7f02c406d770 2023-12-12T09:54:56.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.946+0000 7f02c27fc640 1 -- 172.21.15.72:0/1132047453 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c415b9d0 con 0x7f02c406db50 2023-12-12T09:54:56.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.947+0000 7f02c27fc640 1 --2- 172.21.15.72:0/1132047453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c406db50 0x7f02c4141250 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f02b80097a0 tx=0x7f02b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb491fb12e5f23d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.947+0000 7f02c1ffb640 1 -- 172.21.15.72:0/1132047453 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8031830 con 0x7f02c406db50 2023-12-12T09:54:56.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.947+0000 7f02c1ffb640 1 -- 172.21.15.72:0/1132047453 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b8031990 con 0x7f02c406db50 2023-12-12T09:54:56.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.947+0000 7f02c1ffb640 1 -- 172.21.15.72:0/1132047453 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8031d20 con 0x7f02c406db50 2023-12-12T09:54:56.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.948+0000 7f02c9a00640 1 -- 172.21.15.72:0/1132047453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c406db50 msgr2=0x7f02c4141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.948+0000 7f02c9a00640 1 --2- 172.21.15.72:0/1132047453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c406db50 0x7f02c4141250 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f02b80097a0 tx=0x7f02b802b8d0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.948+0000 7f02c9a00640 1 -- 172.21.15.72:0/1132047453 shutdown_connections 2023-12-12T09:54:56.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.948+0000 7f02c9a00640 1 --2- 172.21.15.72:0/1132047453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c406db50 0x7f02c4141250 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.948+0000 7f02c9a00640 1 -- 172.21.15.72:0/1132047453 >> 172.21.15.72:0/1132047453 conn(0x7f02c40cb850 msgr2=0x7f02c40cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:56.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.948+0000 7f02c9a00640 1 -- 172.21.15.72:0/1132047453 shutdown_connections 2023-12-12T09:54:56.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.948+0000 7f02c9a00640 1 -- 172.21.15.72:0/1132047453 wait complete. 2023-12-12T09:54:56.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.950+0000 7f02c9a00640 1 Processor -- start 2023-12-12T09:54:56.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.950+0000 7f02c9a00640 1 -- start start 2023-12-12T09:54:56.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c9a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c406d770 0x7f02c413c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c406db50 0x7f02c415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c9a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c4141790 0x7f02c415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c9a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c414faa0 con 0x7f02c4141790 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c414ee90 con 0x7f02c406db50 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c9a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c414f170 con 0x7f02c406d770 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c4141790 0x7f02c415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c4141790 0x7f02c415ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53184/0 (socket says 172.21.15.72:53184) 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c37fe640 1 -- 172.21.15.72:0/2474917340 learned_addr learned my addr 172.21.15.72:0/2474917340 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c2ffd640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c406d770 0x7f02c413c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.951+0000 7f02c27fc640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c406db50 0x7f02c415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c37fe640 1 -- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c406d770 msgr2=0x7f02c413c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c37fe640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c406d770 0x7f02c413c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c37fe640 1 -- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c406db50 msgr2=0x7f02c415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:56.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c37fe640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c406db50 0x7f02c415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:56.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c37fe640 1 -- 172.21.15.72:0/2474917340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b8009400 con 0x7f02c4141790 2023-12-12T09:54:56.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c2ffd640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c406d770 0x7f02c413c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:56.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c27fc640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c406db50 0x7f02c415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:56.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c37fe640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c4141790 0x7f02c415ce00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00b970 tx=0x7f02bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02abfff640 1 -- 172.21.15.72:0/2474917340 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00cb40 con 0x7f02c4141790 2023-12-12T09:54:56.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02abfff640 1 -- 172.21.15.72:0/2474917340 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc00cca0 con 0x7f02c4141790 2023-12-12T09:54:56.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02abfff640 1 -- 172.21.15.72:0/2474917340 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc014680 con 0x7f02c4141790 2023-12-12T09:54:56.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c414f450 con 0x7f02c4141790 2023-12-12T09:54:56.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.952+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c413c700 con 0x7f02c4141790 2023-12-12T09:54:56.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.954+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c4067910 con 0x7f02c4141790 2023-12-12T09:54:56.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.954+0000 7f02abfff640 1 -- 172.21.15.72:0/2474917340 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02bc01d3b0 con 0x7f02c4141790 2023-12-12T09:54:56.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.955+0000 7f02abfff640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02b006ae40 0x7f02b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:56.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.955+0000 7f02abfff640 1 -- 172.21.15.72:0/2474917340 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f02bc0895d0 con 0x7f02c4141790 2023-12-12T09:54:56.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.955+0000 7f02c2ffd640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02b006ae40 0x7f02b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:56.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.956+0000 7f02c2ffd640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02b006ae40 0x7f02b006d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f02b4000f60 tx=0x7f02b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:56.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:56.959+0000 7f02abfff640 1 -- 172.21.15.72:0/2474917340 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02bc0544f0 con 0x7f02c4141790 2023-12-12T09:54:57.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.147+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02c40745d0 con 0x7f02c4141790 2023-12-12T09:54:57.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.148+0000 7f02abfff640 1 -- 172.21.15.72:0/2474917340 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02bc057b10 con 0x7f02c4141790 2023-12-12T09:54:57.150 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02b006ae40 msgr2=0x7f02b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02b006ae40 0x7f02b006d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f02b4000f60 tx=0x7f02b4009040 comp rx=0 tx=0).stop 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c4141790 msgr2=0x7f02c415ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c4141790 0x7f02c415ce00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00b970 tx=0x7f02bc00be30 comp rx=0 tx=0).stop 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 shutdown_connections 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f02b006ae40 0x7f02b006d300 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02c4141790 0x7f02c415ce00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c406db50 0x7f02c415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 --2- 172.21.15.72:0/2474917340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c406d770 0x7f02c413c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 >> 172.21.15.72:0/2474917340 conn(0x7f02c40cb850 msgr2=0x7f02c415b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:57.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 shutdown_connections 2023-12-12T09:54:57.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.151+0000 7f02c9a00640 1 -- 172.21.15.72:0/2474917340 wait complete. 2023-12-12T09:54:57.173 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:54:57.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.381+0000 7efc4099a640 1 Processor -- start 2023-12-12T09:54:57.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.381+0000 7efc4099a640 1 -- start start 2023-12-12T09:54:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.383+0000 7efc4099a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc3c0ce150 0x7efc3c0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.383+0000 7efc4099a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c0cedf0 0x7efc3c0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.383+0000 7efc4099a640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c0cf790 con 0x7efc3c155400 2023-12-12T09:54:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.383+0000 7efc4099a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c06c500 con 0x7efc3c0ce150 2023-12-12T09:54:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.383+0000 7efc4099a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15b4a0 con 0x7efc3c0cedf0 2023-12-12T09:54:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.383+0000 7efc3b7fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7efc3c155400 0x7efc3c06bdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36958/0 (socket says 172.21.15.72:36958) 2023-12-12T09:54:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.383+0000 7efc3b7fe640 1 -- 172.21.15.72:0/315298274 learned_addr learned my addr 172.21.15.72:0/315298274 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761239957 0 0) 0x7efc3c0cf790 con 0x7efc3c155400 2023-12-12T09:54:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3affd640 1 --2- 172.21.15.72:0/315298274 >> v2:172.21.15.113:3300/0 conn(0x7efc3c0ce150 0x7efc3c0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3bfff640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c0cedf0 0x7efc3c0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc24003520 con 0x7efc3c155400 2023-12-12T09:54:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1374462295 0 0) 0x7efc24003520 con 0x7efc3c155400 2023-12-12T09:54:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c0cedf0 msgr2=0x7efc3c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c0cedf0 0x7efc3c0cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 >> v2:172.21.15.113:3300/0 conn(0x7efc3c0ce150 msgr2=0x7efc3c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 --2- 172.21.15.72:0/315298274 >> v2:172.21.15.113:3300/0 conn(0x7efc3c0ce150 0x7efc3c0ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0adbb0 con 0x7efc3c155400 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7efc2c001690 con 0x7efc3c155400 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 >> v1:172.21.15.72:6789/0 conn(0x7efc3c155400 legacy=0x7efc3c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c0cedf0 0x7efc24003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc24003ff0 0x7efc240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc24004d00 0x7efc24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c06c500 con 0x7efc24004d00 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15b4a0 con 0x7efc24003ff0 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2400bae0 con 0x7efc3c0cedf0 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3affd640 1 --2- 172.21.15.72:0/315298274 >> v2:172.21.15.113:3300/0 conn(0x7efc3c0ce150 0x7efc3c0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.384+0000 7efc3bfff640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc24004d00 0x7efc24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.385+0000 7efc3b7fe640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c0cedf0 0x7efc24003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.385+0000 7efc3affd640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc24003ff0 0x7efc240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.385+0000 7efc3bfff640 1 -- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c0cedf0 msgr2=0x7efc24003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:57.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.385+0000 7efc3bfff640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c0cedf0 0x7efc24003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.385+0000 7efc3bfff640 1 -- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc24003ff0 msgr2=0x7efc240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:57.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.385+0000 7efc3bfff640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc24003ff0 0x7efc240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.385+0000 7efc3bfff640 1 -- 172.21.15.72:0/315298274 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24003520 con 0x7efc24004d00 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.386+0000 7efc3bfff640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc24004d00 0x7efc24005180 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efc34007360 tx=0x7efc240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a07ebf294214cd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.386+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3402ccd0 con 0x7efc24004d00 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.386+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3402ce30 con 0x7efc24004d00 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.386+0000 7efc3a7fc640 1 -- 172.21.15.72:0/315298274 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34034660 con 0x7efc24004d00 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.387+0000 7efc4099a640 1 -- 172.21.15.72:0/315298274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc24004d00 msgr2=0x7efc24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.387+0000 7efc4099a640 1 --2- 172.21.15.72:0/315298274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc24004d00 0x7efc24005180 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efc34007360 tx=0x7efc240057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.387+0000 7efc3bfff640 1 -- 172.21.15.72:0/315298274 reap_dead start 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.387+0000 7efc4099a640 1 -- 172.21.15.72:0/315298274 shutdown_connections 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.387+0000 7efc4099a640 1 -- 172.21.15.72:0/315298274 >> 172.21.15.72:0/315298274 conn(0x7efc3c06a640 msgr2=0x7efc3c06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.387+0000 7efc4099a640 1 -- 172.21.15.72:0/315298274 shutdown_connections 2023-12-12T09:54:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.387+0000 7efc4099a640 1 -- 172.21.15.72:0/315298274 wait complete. 2023-12-12T09:54:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.389+0000 7efc4099a640 1 Processor -- start 2023-12-12T09:54:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.389+0000 7efc4099a640 1 -- start start 2023-12-12T09:54:57.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc4099a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc24004d00 0x7efc3c09b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc4099a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc3c156760 0x7efc3c1ef920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc4099a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c1efe60 0x7efc3c1f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc4099a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15a2d0 con 0x7efc3c156760 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc4099a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1f4a80 con 0x7efc3c1efe60 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc4099a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1f0310 con 0x7efc24004d00 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc3affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc3c156760 0x7efc3c1ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc3affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc3c156760 0x7efc3c1ef920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53198/0 (socket says 172.21.15.72:53198) 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc3affd640 1 -- 172.21.15.72:0/194215586 learned_addr learned my addr 172.21.15.72:0/194215586 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc3b7fe640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc24004d00 0x7efc3c09b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc3bfff640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c1efe60 0x7efc3c1f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc3affd640 1 -- 172.21.15.72:0/194215586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc24004d00 msgr2=0x7efc3c09b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.390+0000 7efc3affd640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc24004d00 0x7efc3c09b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc3affd640 1 -- 172.21.15.72:0/194215586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c1efe60 msgr2=0x7efc3c1f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc3affd640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c1efe60 0x7efc3c1f3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc3affd640 1 -- 172.21.15.72:0/194215586 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c1f47a0 con 0x7efc3c156760 2023-12-12T09:54:57.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc3affd640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc3c156760 0x7efc3c1ef920 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efc30010860 tx=0x7efc30010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:57.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc23fff640 1 -- 172.21.15.72:0/194215586 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000cb40 con 0x7efc3c156760 2023-12-12T09:54:57.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc23fff640 1 -- 172.21.15.72:0/194215586 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3000cca0 con 0x7efc3c156760 2023-12-12T09:54:57.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c155c80 con 0x7efc3c156760 2023-12-12T09:54:57.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc23fff640 1 -- 172.21.15.72:0/194215586 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30015690 con 0x7efc3c156760 2023-12-12T09:54:57.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.391+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c156240 con 0x7efc3c156760 2023-12-12T09:54:57.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.392+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c155ae0 con 0x7efc3c156760 2023-12-12T09:54:57.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.393+0000 7efc23fff640 1 -- 172.21.15.72:0/194215586 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc300157f0 con 0x7efc3c156760 2023-12-12T09:54:57.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.394+0000 7efc23fff640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc2806ae40 0x7efc2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.394+0000 7efc23fff640 1 -- 172.21.15.72:0/194215586 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7efc30089d30 con 0x7efc3c156760 2023-12-12T09:54:57.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.395+0000 7efc3b7fe640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc2806ae40 0x7efc2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.395+0000 7efc3b7fe640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc2806ae40 0x7efc2806d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efc3c06cf60 tx=0x7efc2c000f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:57.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.399+0000 7efc23fff640 1 -- 172.21.15.72:0/194215586 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc3c155ae0 con 0x7efc3c156760 2023-12-12T09:54:57.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.594+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc3c155ae0 con 0x7efc3c156760 2023-12-12T09:54:57.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.595+0000 7efc23fff640 1 -- 172.21.15.72:0/194215586 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc3c155ae0 con 0x7efc3c156760 2023-12-12T09:54:57.597 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2023-12-12T09:54:57.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc2806ae40 msgr2=0x7efc2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc2806ae40 0x7efc2806d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efc3c06cf60 tx=0x7efc2c000f50 comp rx=0 tx=0).stop 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc3c156760 msgr2=0x7efc3c1ef920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc3c156760 0x7efc3c1ef920 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efc30010860 tx=0x7efc30010d20 comp rx=0 tx=0).stop 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 shutdown_connections 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efc2806ae40 0x7efc2806d300 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c1efe60 0x7efc3c1f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc3c156760 0x7efc3c1ef920 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 --2- 172.21.15.72:0/194215586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc24004d00 0x7efc3c09b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 >> 172.21.15.72:0/194215586 conn(0x7efc3c06a640 msgr2=0x7efc3c15dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 shutdown_connections 2023-12-12T09:54:57.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.598+0000 7efc4099a640 1 -- 172.21.15.72:0/194215586 wait complete. 2023-12-12T09:54:57.620 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:57.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.829+0000 7f8f9aec9640 1 Processor -- start 2023-12-12T09:54:57.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.829+0000 7f8f9aec9640 1 -- start start 2023-12-12T09:54:57.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.831+0000 7f8f9aec9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8f94157a60 0x7f8f9406c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.831+0000 7f8f9aec9640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9406ca40 con 0x7f8f9415e2b0 2023-12-12T09:54:57.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.831+0000 7f8f9aec9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f94071cd0 con 0x7f8f9406de20 2023-12-12T09:54:57.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.831+0000 7f8f9aec9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f94070f10 con 0x7f8f94157a60 2023-12-12T09:54:57.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.831+0000 7f8f99466640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f8f9415e2b0 0x7f8f940cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36964/0 (socket says 172.21.15.72:36964) 2023-12-12T09:54:57.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.831+0000 7f8f99466640 1 -- 172.21.15.72:0/920089313 learned_addr learned my addr 172.21.15.72:0/920089313 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:57.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.831+0000 7f8f98c65640 1 --2- 172.21.15.72:0/920089313 >> v2:172.21.15.113:3300/0 conn(0x7f8f94157a60 0x7f8f9406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308952715 0 0) 0x7f8f9406ca40 con 0x7f8f9415e2b0 2023-12-12T09:54:57.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f84003520 con 0x7f8f9415e2b0 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3199689820 0 0) 0x7f8f84003520 con 0x7f8f9415e2b0 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 >> v2:172.21.15.113:3300/0 conn(0x7f8f94157a60 msgr2=0x7f8f9406c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 --2- 172.21.15.72:0/920089313 >> v2:172.21.15.113:3300/0 conn(0x7f8f94157a60 0x7f8f9406c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 >> v1:172.21.15.113:6789/0 conn(0x7f8f9406de20 legacy=0x7f8f9415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940711f0 con 0x7f8f9415e2b0 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f8f8c0019f0 con 0x7f8f9415e2b0 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 >> v1:172.21.15.72:6789/0 conn(0x7f8f9415e2b0 legacy=0x7f8f940cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f84003970 0x7f8f84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f840047c0 0x7f8f84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f84005180 0x7f8f8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94070f10 con 0x7f8f840047c0 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8400c120 con 0x7f8f84005180 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.832+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8400c2e0 con 0x7f8f84003970 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f93fff640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f840047c0 0x7f8f84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f99466640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f84005180 0x7f8f8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f98c65640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f84003970 0x7f8f84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f93fff640 1 -- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f84003970 msgr2=0x7f8f84003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f93fff640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f84003970 0x7f8f84003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f93fff640 1 -- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f84005180 msgr2=0x7f8f8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f93fff640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f84005180 0x7f8f8400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f93fff640 1 -- 172.21.15.72:0/920089313 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f84003520 con 0x7f8f840047c0 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f99466640 1 -- 172.21.15.72:0/920089313 reap_dead start 2023-12-12T09:54:57.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.833+0000 7f8f98c65640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f84003970 0x7f8f84003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:57.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.834+0000 7f8f93fff640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f840047c0 0x7f8f84004c40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8f80001750 tx=0x7f8f840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7687e2d3cc92f19 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:57.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.834+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800052d0 con 0x7f8f840047c0 2023-12-12T09:54:57.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.834+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8002e050 con 0x7f8f840047c0 2023-12-12T09:54:57.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.834+0000 7f8f937fe640 1 -- 172.21.15.72:0/920089313 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80003830 con 0x7f8f840047c0 2023-12-12T09:54:57.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.835+0000 7f8f9aec9640 1 -- 172.21.15.72:0/920089313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f840047c0 msgr2=0x7f8f84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.835+0000 7f8f9aec9640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f840047c0 0x7f8f84004c40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8f80001750 tx=0x7f8f840057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.835+0000 7f8f9aec9640 1 -- 172.21.15.72:0/920089313 shutdown_connections 2023-12-12T09:54:57.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.835+0000 7f8f9aec9640 1 --2- 172.21.15.72:0/920089313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f840047c0 0x7f8f84004c40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.835+0000 7f8f9aec9640 1 -- 172.21.15.72:0/920089313 >> 172.21.15.72:0/920089313 conn(0x7f8f940b0a90 msgr2=0x7f8f940b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:57.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.835+0000 7f8f9aec9640 1 -- 172.21.15.72:0/920089313 shutdown_connections 2023-12-12T09:54:57.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.835+0000 7f8f9aec9640 1 -- 172.21.15.72:0/920089313 wait complete. 2023-12-12T09:54:57.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.837+0000 7f8f9aec9640 1 Processor -- start 2023-12-12T09:54:57.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.837+0000 7f8f9aec9640 1 -- start start 2023-12-12T09:54:57.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f9aec9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840047c0 0x7f8f940b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f9aec9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940b4910 0x7f8f940cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f9aec9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f940b4d90 0x7f8f940cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f9aec9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94152eb0 con 0x7f8f940b4d90 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f9aec9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94152000 con 0x7f8f940b4910 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f9aec9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f941522a0 con 0x7f8f840047c0 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f99466640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f940b4d90 0x7f8f940cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f99466640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f940b4d90 0x7f8f940cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53220/0 (socket says 172.21.15.72:53220) 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.838+0000 7f8f99466640 1 -- 172.21.15.72:0/470507799 learned_addr learned my addr 172.21.15.72:0/470507799 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f98c65640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840047c0 0x7f8f940b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f93fff640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940b4910 0x7f8f940cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f99466640 1 -- 172.21.15.72:0/470507799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840047c0 msgr2=0x7f8f940b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f99466640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840047c0 0x7f8f940b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f99466640 1 -- 172.21.15.72:0/470507799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940b4910 msgr2=0x7f8f940cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f99466640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940b4910 0x7f8f940cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f99466640 1 -- 172.21.15.72:0/470507799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94152580 con 0x7f8f940b4d90 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f99466640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f940b4d90 0x7f8f940cde60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8f94070150 tx=0x7f8f8c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:57.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f917fa640 1 -- 172.21.15.72:0/470507799 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00eb90 con 0x7f8f940b4d90 2023-12-12T09:54:57.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f917fa640 1 -- 172.21.15.72:0/470507799 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c00ecf0 con 0x7f8f940b4d90 2023-12-12T09:54:57.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f917fa640 1 -- 172.21.15.72:0/470507799 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c013660 con 0x7f8f940b4d90 2023-12-12T09:54:57.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f94152860 con 0x7f8f940b4d90 2023-12-12T09:54:57.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.839+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f940b33e0 con 0x7f8f940b4d90 2023-12-12T09:54:57.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.841+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f94139370 con 0x7f8f940b4d90 2023-12-12T09:54:57.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.841+0000 7f8f917fa640 1 -- 172.21.15.72:0/470507799 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f8c0137c0 con 0x7f8f940b4d90 2023-12-12T09:54:57.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.842+0000 7f8f917fa640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8f7806ae70 0x7f8f7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:57.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.842+0000 7f8f917fa640 1 -- 172.21.15.72:0/470507799 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f8f8c089b70 con 0x7f8f940b4d90 2023-12-12T09:54:57.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.843+0000 7f8f98c65640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8f7806ae70 0x7f8f7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:57.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.843+0000 7f8f98c65640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8f7806ae70 0x7f8f7806d330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f940ce730 tx=0x7f8f88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:57.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:57.847+0000 7f8f917fa640 1 -- 172.21.15.72:0/470507799 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f8c058490 con 0x7f8f940b4d90 2023-12-12T09:54:58.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.077+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f9415d8d0 con 0x7f8f940b4d90 2023-12-12T09:54:58.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.078+0000 7f8f917fa640 1 -- 172.21.15.72:0/470507799 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f8f8c058020 con 0x7f8f940b4d90 2023-12-12T09:54:58.080 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:58.080 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/130800926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":130800926},{"type":"v1","addr":"172.21.15.72:6836","nonce":130800926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/318582100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":318582100},{"type":"v1","addr":"172.21.15.72:6837","nonce":318582100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2380516950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2380516950},{"type":"v1","addr":"172.21.15.113:6835","nonce":2380516950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:54:32.882443+0000","modified":"2023-12-12T09:54:35.940697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/4181915308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4181915308},{"type":"v1","addr":"172.21.15.113:6834","nonce":4181915308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-12T09:54:58.081 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 25 2023-12-12T09:54:58.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8f7806ae70 msgr2=0x7f8f7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8f7806ae70 0x7f8f7806d330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f940ce730 tx=0x7f8f88008040 comp rx=0 tx=0).stop 2023-12-12T09:54:58.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f940b4d90 msgr2=0x7f8f940cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f940b4d90 0x7f8f940cde60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8f94070150 tx=0x7f8f8c00e530 comp rx=0 tx=0).stop 2023-12-12T09:54:58.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 shutdown_connections 2023-12-12T09:54:58.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f8f7806ae70 0x7f8f7806d330 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f940b4d90 0x7f8f940cde60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940b4910 0x7f8f940cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 --2- 172.21.15.72:0/470507799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f840047c0 0x7f8f940b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 >> 172.21.15.72:0/470507799 conn(0x7f8f940b0a90 msgr2=0x7f8f94154450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:58.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 shutdown_connections 2023-12-12T09:54:58.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.081+0000 7f8f9aec9640 1 -- 172.21.15.72:0/470507799 wait complete. 2023-12-12T09:54:58.101 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:58.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.303+0000 7fee14f05640 1 Processor -- start 2023-12-12T09:54:58.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.303+0000 7fee14f05640 1 -- start start 2023-12-12T09:54:58.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.305+0000 7fee14f05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10072180 0x7fee100b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.305+0000 7fee14f05640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee100b3bb0 con 0x7fee10143e00 2023-12-12T09:54:58.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.305+0000 7fee14f05640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee101441e0 con 0x7fee1006c410 2023-12-12T09:54:58.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.305+0000 7fee14f05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee10072680 con 0x7fee10072180 2023-12-12T09:54:58.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.305+0000 7fee0e59c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fee10143e00 0x7fee100b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36974/0 (socket says 172.21.15.72:36974) 2023-12-12T09:54:58.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.305+0000 7fee0e59c640 1 -- 172.21.15.72:0/596454444 learned_addr learned my addr 172.21.15.72:0/596454444 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:58.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107168406 0 0) 0x7fee100b3bb0 con 0x7fee10143e00 2023-12-12T09:54:58.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0dd9b640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10072180 0x7fee100b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedfc003520 con 0x7fee10143e00 2023-12-12T09:54:58.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3493818374 0 0) 0x7fedfc003520 con 0x7fee10143e00 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10072180 msgr2=0x7fee100b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10072180 0x7fee100b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 >> v1:172.21.15.113:6789/0 conn(0x7fee1006c410 legacy=0x7fee1015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100b1e30 con 0x7fee10143e00 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fedf8001670 con 0x7fee10143e00 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 >> v1:172.21.15.72:6789/0 conn(0x7fee10143e00 legacy=0x7fee100b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedfc0039a0 0x7fedfc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fedfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10072180 0x7fedfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10072680 con 0x7fedfc0047c0 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc00bab0 con 0x7fedfc0039a0 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.306+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc00bc70 con 0x7fee10072180 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.307+0000 7fee0dd9b640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fedfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.307+0000 7fee0e59c640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedfc0039a0 0x7fedfc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.307+0000 7fee0ed9d640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10072180 0x7fedfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.307+0000 7fee0dd9b640 1 -- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10072180 msgr2=0x7fedfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.307+0000 7fee0dd9b640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10072180 0x7fedfc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.307+0000 7fee0dd9b640 1 -- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedfc0039a0 msgr2=0x7fedfc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.307+0000 7fee0dd9b640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedfc0039a0 0x7fedfc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.307+0000 7fee0dd9b640 1 -- 172.21.15.72:0/596454444 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc003520 con 0x7fedfc0047c0 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.308+0000 7fee0dd9b640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fedfc004c40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fee00007360 tx=0x7fedfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e49360157cd13b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.308+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0002ccd0 con 0x7fedfc0047c0 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0002ce30 con 0x7fedfc0047c0 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee0d59a640 1 -- 172.21.15.72:0/596454444 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00034660 con 0x7fedfc0047c0 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee14f05640 1 -- 172.21.15.72:0/596454444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 msgr2=0x7fedfc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee14f05640 1 --2- 172.21.15.72:0/596454444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fedfc004c40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fee00007360 tx=0x7fedfc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee0ed9d640 1 -- 172.21.15.72:0/596454444 reap_dead start 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee14f05640 1 -- 172.21.15.72:0/596454444 shutdown_connections 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee14f05640 1 -- 172.21.15.72:0/596454444 >> 172.21.15.72:0/596454444 conn(0x7fee100b0a90 msgr2=0x7fee100b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee14f05640 1 -- 172.21.15.72:0/596454444 shutdown_connections 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.309+0000 7fee14f05640 1 -- 172.21.15.72:0/596454444 wait complete. 2023-12-12T09:54:58.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.311+0000 7fee14f05640 1 Processor -- start 2023-12-12T09:54:58.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.311+0000 7fee14f05640 1 -- start start 2023-12-12T09:54:58.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee14f05640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fee101364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee14f05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10142520 0x7fee1013fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee14f05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee10140200 0x7fee101406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee14f05640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1015a540 con 0x7fedfc0047c0 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee14f05640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10141480 con 0x7fee10140200 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee14f05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee10141760 con 0x7fee10142520 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee0e59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fee101364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee0e59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fee101364a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53232/0 (socket says 172.21.15.72:53232) 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.312+0000 7fee0e59c640 1 -- 172.21.15.72:0/3816564168 learned_addr learned my addr 172.21.15.72:0/3816564168 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee0e59c640 1 -- 172.21.15.72:0/3816564168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10142520 msgr2=0x7fee1013fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee0e59c640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10142520 0x7fee1013fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee0e59c640 1 -- 172.21.15.72:0/3816564168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee10140200 msgr2=0x7fee101406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee0e59c640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee10140200 0x7fee101406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee0e59c640 1 -- 172.21.15.72:0/3816564168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee10141a40 con 0x7fedfc0047c0 2023-12-12T09:54:58.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee0e59c640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fee101364a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fedf800e8c0 tx=0x7fedf800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:58.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee06ffd640 1 -- 172.21.15.72:0/3816564168 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800c8d0 con 0x7fedfc0047c0 2023-12-12T09:54:58.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee06ffd640 1 -- 172.21.15.72:0/3816564168 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf800ca30 con 0x7fedfc0047c0 2023-12-12T09:54:58.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee06ffd640 1 -- 172.21.15.72:0/3816564168 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800cde0 con 0x7fedfc0047c0 2023-12-12T09:54:58.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee10141d20 con 0x7fedfc0047c0 2023-12-12T09:54:58.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.313+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1015ad90 con 0x7fedfc0047c0 2023-12-12T09:54:58.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.314+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee10142240 con 0x7fedfc0047c0 2023-12-12T09:54:58.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.315+0000 7fee06ffd640 1 -- 172.21.15.72:0/3816564168 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fedf801d460 con 0x7fedfc0047c0 2023-12-12T09:54:58.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.316+0000 7fee06ffd640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fedf006ae40 0x7fedf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.316+0000 7fee06ffd640 1 -- 172.21.15.72:0/3816564168 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fedf8089860 con 0x7fedfc0047c0 2023-12-12T09:54:58.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.316+0000 7fee0dd9b640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fedf006ae40 0x7fedf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.317+0000 7fee0dd9b640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fedf006ae40 0x7fedf006d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fee00036000 tx=0x7fee0002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:58.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.320+0000 7fee06ffd640 1 -- 172.21.15.72:0/3816564168 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee10142240 con 0x7fedfc0047c0 2023-12-12T09:54:58.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.511+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee10142240 con 0x7fedfc0047c0 2023-12-12T09:54:58.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.514+0000 7fee06ffd640 1 -- 172.21.15.72:0/3816564168 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16812 (secure 0 0 0) 0x7fee10142240 con 0x7fedfc0047c0 2023-12-12T09:54:58.516 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:58.516 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":53,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:56.470479+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:54:30.694215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-12T09:54:31.894291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.c","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:58.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fedf006ae40 msgr2=0x7fedf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fedf006ae40 0x7fedf006d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fee00036000 tx=0x7fee0002e000 comp rx=0 tx=0).stop 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 msgr2=0x7fee101364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fee101364a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fedf800e8c0 tx=0x7fedf800ed80 comp rx=0 tx=0).stop 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 shutdown_connections 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fedf006ae40 0x7fedf006d300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee10140200 0x7fee101406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee10142520 0x7fee1013fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 --2- 172.21.15.72:0/3816564168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fedfc0047c0 0x7fee101364a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.516+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 >> 172.21.15.72:0/3816564168 conn(0x7fee100b0a90 msgr2=0x7fee1006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.517+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 shutdown_connections 2023-12-12T09:54:58.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.517+0000 7fee14f05640 1 -- 172.21.15.72:0/3816564168 wait complete. 2023-12-12T09:54:58.538 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-12T09:54:58.538 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:58.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.739+0000 7f364f13c640 1 Processor -- start 2023-12-12T09:54:58.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.739+0000 7f364f13c640 1 -- start start 2023-12-12T09:54:58.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.741+0000 7f364f13c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f364815a0d0 0x7f364806df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.741+0000 7f364f13c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36480cf1c0 0x7f36480b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.741+0000 7f364f13c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480cda10 con 0x7f364806e470 2023-12-12T09:54:58.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.741+0000 7f364f13c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36480cdcf0 con 0x7f364815a0d0 2023-12-12T09:54:58.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.741+0000 7f364f13c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480cdfd0 con 0x7f36480cf1c0 2023-12-12T09:54:58.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f364ced8640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f364815a0d0 0x7f364806df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f3647fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f364806e470 0x7f36480ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36986/0 (socket says 172.21.15.72:36986) 2023-12-12T09:54:58.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f3647fff640 1 -- 172.21.15.72:0/2152580548 learned_addr learned my addr 172.21.15.72:0/2152580548 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:58.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f36477fe640 1 -- 172.21.15.72:0/2152580548 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169247665 0 0) 0x7f36480cda10 con 0x7f364806e470 2023-12-12T09:54:58.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f36477fe640 1 -- 172.21.15.72:0/2152580548 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3634005730 con 0x7f364806e470 2023-12-12T09:54:58.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f364d6d9640 1 --2- 172.21.15.72:0/2152580548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36480cf1c0 0x7f36480b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f364ced8640 1 -- 172.21.15.72:0/2152580548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36480cf1c0 msgr2=0x7f36480b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:58.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f364ced8640 1 --2- 172.21.15.72:0/2152580548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36480cf1c0 0x7f36480b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f364ced8640 1 -- 172.21.15.72:0/2152580548 >> v1:172.21.15.72:6789/0 conn(0x7f364806e470 legacy=0x7f36480ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.742+0000 7f364ced8640 1 -- 172.21.15.72:0/2152580548 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480ce2b0 con 0x7f364815a0d0 2023-12-12T09:54:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.744+0000 7f364ced8640 1 --2- 172.21.15.72:0/2152580548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364806df30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3638009d90 tx=0x7f363802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bf86a30b462254 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.744+0000 7f36477fe640 1 -- 172.21.15.72:0/2152580548 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363802cbc0 con 0x7f364815a0d0 2023-12-12T09:54:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.744+0000 7f36477fe640 1 -- 172.21.15.72:0/2152580548 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363802cd20 con 0x7f364815a0d0 2023-12-12T09:54:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.744+0000 7f36477fe640 1 -- 172.21.15.72:0/2152580548 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638034660 con 0x7f364815a0d0 2023-12-12T09:54:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.745+0000 7f364f13c640 1 -- 172.21.15.72:0/2152580548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 msgr2=0x7f364806df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.745+0000 7f364f13c640 1 --2- 172.21.15.72:0/2152580548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364806df30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3638009d90 tx=0x7f363802be10 comp rx=0 tx=0).stop 2023-12-12T09:54:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.745+0000 7f364f13c640 1 -- 172.21.15.72:0/2152580548 shutdown_connections 2023-12-12T09:54:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.745+0000 7f364f13c640 1 --2- 172.21.15.72:0/2152580548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36480cf1c0 0x7f36480b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.745+0000 7f364f13c640 1 --2- 172.21.15.72:0/2152580548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364806df30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.745+0000 7f364f13c640 1 -- 172.21.15.72:0/2152580548 >> 172.21.15.72:0/2152580548 conn(0x7f36480cfe30 msgr2=0x7f36480d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.745+0000 7f364f13c640 1 -- 172.21.15.72:0/2152580548 shutdown_connections 2023-12-12T09:54:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.745+0000 7f364f13c640 1 -- 172.21.15.72:0/2152580548 wait complete. 2023-12-12T09:54:58.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.747+0000 7f364f13c640 1 Processor -- start 2023-12-12T09:54:58.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.747+0000 7f364f13c640 1 -- start start 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364f13c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364806e470 0x7f3648146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364f13c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36480cf1c0 0x7f364814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364f13c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364814fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364f13c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648133920 con 0x7f364815a0d0 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364f13c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648147250 con 0x7f36480cf1c0 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364f13c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36481332f0 con 0x7f364806e470 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364d6d9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364814fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364d6d9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364814fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53256/0 (socket says 172.21.15.72:53256) 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364d6d9640 1 -- 172.21.15.72:0/404202281 learned_addr learned my addr 172.21.15.72:0/404202281 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364d6d9640 1 -- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364806e470 msgr2=0x7f3648146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364ced8640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364806e470 0x7f3648146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364d6d9640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364806e470 0x7f3648146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f3647fff640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36480cf1c0 0x7f364814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364d6d9640 1 -- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36480cf1c0 msgr2=0x7f364814f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364d6d9640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36480cf1c0 0x7f364814f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.748+0000 7f364d6d9640 1 -- 172.21.15.72:0/404202281 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009a70 con 0x7f364815a0d0 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.749+0000 7f3647fff640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36480cf1c0 0x7f364814f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.749+0000 7f364d6d9640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364814fd50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f364000c5f0 tx=0x7f364000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.749+0000 7f36457fa640 1 -- 172.21.15.72:0/404202281 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640004220 con 0x7f364815a0d0 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.749+0000 7f36457fa640 1 -- 172.21.15.72:0/404202281 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640004380 con 0x7f364815a0d0 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.749+0000 7f36457fa640 1 -- 172.21.15.72:0/404202281 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640003850 con 0x7f364815a0d0 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.749+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364815bd40 con 0x7f364815a0d0 2023-12-12T09:54:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.749+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3648133540 con 0x7f364815a0d0 2023-12-12T09:54:58.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.750+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3648133a60 con 0x7f364815a0d0 2023-12-12T09:54:58.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.751+0000 7f36457fa640 1 -- 172.21.15.72:0/404202281 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36400039b0 con 0x7f364815a0d0 2023-12-12T09:54:58.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.752+0000 7f36457fa640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae70 0x7f362806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:58.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.752+0000 7f36457fa640 1 -- 172.21.15.72:0/404202281 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f3640089410 con 0x7f364815a0d0 2023-12-12T09:54:58.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.753+0000 7f364ced8640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae70 0x7f362806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:58.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.753+0000 7f364ced8640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae70 0x7f362806d330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3638036000 tx=0x7f3638030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:58.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.757+0000 7f36457fa640 1 -- 172.21.15.72:0/404202281 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3648133a60 con 0x7f364815a0d0 2023-12-12T09:54:58.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.968+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3648133a60 con 0x7f364815a0d0 2023-12-12T09:54:58.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.969+0000 7f36457fa640 1 -- 172.21.15.72:0/404202281 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f3648133a60 con 0x7f364815a0d0 2023-12-12T09:54:58.972 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:58.972 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/130800926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":130800926},{"type":"v1","addr":"172.21.15.72:6836","nonce":130800926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/318582100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":318582100},{"type":"v1","addr":"172.21.15.72:6837","nonce":318582100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2380516950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2380516950},{"type":"v1","addr":"172.21.15.113:6835","nonce":2380516950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:54:32.882443+0000","modified":"2023-12-12T09:54:35.940697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/4181915308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4181915308},{"type":"v1","addr":"172.21.15.113:6834","nonce":4181915308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-12T09:54:58.972 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 25 2023-12-12T09:54:58.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae70 msgr2=0x7f362806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae70 0x7f362806d330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3638036000 tx=0x7f3638030000 comp rx=0 tx=0).stop 2023-12-12T09:54:58.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 msgr2=0x7f364814fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:58.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364814fd50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f364000c5f0 tx=0x7f364000cab0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 shutdown_connections 2023-12-12T09:54:58.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f362806ae70 0x7f362806d330 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f364815a0d0 0x7f364814fd50 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36480cf1c0 0x7f364814f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 --2- 172.21.15.72:0/404202281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364806e470 0x7f3648146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:58.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 >> 172.21.15.72:0/404202281 conn(0x7f36480cfe30 msgr2=0x7f36480cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:58.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 shutdown_connections 2023-12-12T09:54:58.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:58.972+0000 7f364f13c640 1 -- 172.21.15.72:0/404202281 wait complete. 2023-12-12T09:54:58.991 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:54:59.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.193+0000 7fd4e4eec640 1 Processor -- start 2023-12-12T09:54:59.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.193+0000 7fd4e4eec640 1 -- start start 2023-12-12T09:54:59.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.194+0000 7fd4e4eec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00cd000 0x7fd4e006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.194+0000 7fd4e4eec640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e006d510 con 0x7fd4e01467c0 2023-12-12T09:54:59.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.194+0000 7fd4e4eec640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e006d7f0 con 0x7fd4e0146ba0 2023-12-12T09:54:59.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.194+0000 7fd4e4eec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e00ac860 con 0x7fd4e00cd000 2023-12-12T09:54:59.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.195+0000 7fd4de59c640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd4e01467c0 0x7fd4e00cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36988/0 (socket says 172.21.15.72:36988) 2023-12-12T09:54:59.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.195+0000 7fd4de59c640 1 -- 172.21.15.72:0/2595249334 learned_addr learned my addr 172.21.15.72:0/2595249334 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:54:59.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.195+0000 7fd4d7fff640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00cd000 0x7fd4e006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.195+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571303022 0 0) 0x7fd4e006d510 con 0x7fd4e01467c0 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.195+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4cc003520 con 0x7fd4e01467c0 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2840172756 0 0) 0x7fd4cc003520 con 0x7fd4e01467c0 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00cd000 msgr2=0x7fd4e006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00cd000 0x7fd4e006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 >> v1:172.21.15.113:6789/0 conn(0x7fd4e0146ba0 legacy=0x7fd4e006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e006dad0 con 0x7fd4e01467c0 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fd4d0001670 con 0x7fd4e01467c0 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 >> v1:172.21.15.72:6789/0 conn(0x7fd4e01467c0 legacy=0x7fd4e00cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc0039a0 0x7fd4cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00cd000 0x7fd4cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4cc004d50 0x7fd4cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e00ac860 con 0x7fd4cc004d50 2023-12-12T09:54:59.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc00bb40 con 0x7fd4cc0039a0 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc00bd00 con 0x7fd4e00cd000 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4ded9d640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4cc004d50 0x7fd4cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4de59c640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc0039a0 0x7fd4cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.196+0000 7fd4d7fff640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00cd000 0x7fd4cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.197+0000 7fd4ded9d640 1 -- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00cd000 msgr2=0x7fd4cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.197+0000 7fd4ded9d640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00cd000 0x7fd4cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.197+0000 7fd4ded9d640 1 -- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc0039a0 msgr2=0x7fd4cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.197+0000 7fd4ded9d640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc0039a0 0x7fd4cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.197+0000 7fd4ded9d640 1 -- 172.21.15.72:0/2595249334 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc003520 con 0x7fd4cc004d50 2023-12-12T09:54:59.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.198+0000 7fd4ded9d640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4cc004d50 0x7fd4cc0051d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8001e20 tx=0x7fd4cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed1f15ff7349913 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:59.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.198+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8003b70 con 0x7fd4cc004d50 2023-12-12T09:54:59.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.198+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d8003cd0 con 0x7fd4cc004d50 2023-12-12T09:54:59.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.198+0000 7fd4ddd9b640 1 -- 172.21.15.72:0/2595249334 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8005310 con 0x7fd4cc004d50 2023-12-12T09:54:59.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.198+0000 7fd4e4eec640 1 -- 172.21.15.72:0/2595249334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4cc004d50 msgr2=0x7fd4cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.198+0000 7fd4e4eec640 1 --2- 172.21.15.72:0/2595249334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4cc004d50 0x7fd4cc0051d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8001e20 tx=0x7fd4cc0057f0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.198+0000 7fd4ded9d640 1 -- 172.21.15.72:0/2595249334 reap_dead start 2023-12-12T09:54:59.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.199+0000 7fd4e4eec640 1 -- 172.21.15.72:0/2595249334 shutdown_connections 2023-12-12T09:54:59.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.199+0000 7fd4e4eec640 1 -- 172.21.15.72:0/2595249334 >> 172.21.15.72:0/2595249334 conn(0x7fd4e00b0a90 msgr2=0x7fd4e00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:59.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.199+0000 7fd4e4eec640 1 -- 172.21.15.72:0/2595249334 shutdown_connections 2023-12-12T09:54:59.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.199+0000 7fd4e4eec640 1 -- 172.21.15.72:0/2595249334 wait complete. 2023-12-12T09:54:59.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.200+0000 7fd4e4eec640 1 Processor -- start 2023-12-12T09:54:59.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.201+0000 7fd4e4eec640 1 -- start start 2023-12-12T09:54:59.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.201+0000 7fd4e4eec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc004d50 0x7fd4e013f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4e4eec640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4e015fc20 0x7fd4e00b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4e4eec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00b3f00 0x7fd4e00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4e4eec640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0143710 con 0x7fd4e015fc20 2023-12-12T09:54:59.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4e4eec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0142f00 con 0x7fd4cc004d50 2023-12-12T09:54:59.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4e4eec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e015f460 con 0x7fd4e00b3f00 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4e015fc20 0x7fd4e00b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4e015fc20 0x7fd4e00b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53260/0 (socket says 172.21.15.72:53260) 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4d7fff640 1 -- 172.21.15.72:0/3993107483 learned_addr learned my addr 172.21.15.72:0/3993107483 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4ded9d640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00b3f00 0x7fd4e00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4de59c640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc004d50 0x7fd4e013f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4d7fff640 1 -- 172.21.15.72:0/3993107483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00b3f00 msgr2=0x7fd4e00b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4d7fff640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00b3f00 0x7fd4e00b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4d7fff640 1 -- 172.21.15.72:0/3993107483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc004d50 msgr2=0x7fd4e013f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4d7fff640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc004d50 0x7fd4e013f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.202+0000 7fd4d7fff640 1 -- 172.21.15.72:0/3993107483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0143150 con 0x7fd4e015fc20 2023-12-12T09:54:59.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.203+0000 7fd4d7fff640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4e015fc20 0x7fd4e00b23c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800f580 tx=0x7fd4c800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:59.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.203+0000 7fd4d6ffd640 1 -- 172.21.15.72:0/3993107483 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8014870 con 0x7fd4e015fc20 2023-12-12T09:54:59.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.203+0000 7fd4d6ffd640 1 -- 172.21.15.72:0/3993107483 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c80149d0 con 0x7fd4e015fc20 2023-12-12T09:54:59.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.203+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e0143430 con 0x7fd4e015fc20 2023-12-12T09:54:59.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.203+0000 7fd4d6ffd640 1 -- 172.21.15.72:0/3993107483 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8014d80 con 0x7fd4e015fc20 2023-12-12T09:54:59.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.203+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e00b48f0 con 0x7fd4e015fc20 2023-12-12T09:54:59.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.204+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e015f290 con 0x7fd4e015fc20 2023-12-12T09:54:59.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.205+0000 7fd4d6ffd640 1 -- 172.21.15.72:0/3993107483 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4c8015410 con 0x7fd4e015fc20 2023-12-12T09:54:59.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.206+0000 7fd4d6ffd640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd4c006ae40 0x7fd4c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.206+0000 7fd4d6ffd640 1 -- 172.21.15.72:0/3993107483 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fd4c808a220 con 0x7fd4e015fc20 2023-12-12T09:54:59.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.206+0000 7fd4de59c640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd4c006ae40 0x7fd4c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.207+0000 7fd4de59c640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd4c006ae40 0x7fd4c006d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0001eb0 tx=0x7fd4d0000f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:59.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.210+0000 7fd4d6ffd640 1 -- 172.21.15.72:0/3993107483 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4c80550c0 con 0x7fd4e015fc20 2023-12-12T09:54:59.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.436+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4e00b0870 con 0x7fd4e015fc20 2023-12-12T09:54:59.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.437+0000 7fd4d6ffd640 1 -- 172.21.15.72:0/3993107483 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd4c80586e0 con 0x7fd4e015fc20 2023-12-12T09:54:59.439 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:59.439 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/130800926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":130800926},{"type":"v1","addr":"172.21.15.72:6836","nonce":130800926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/318582100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":318582100},{"type":"v1","addr":"172.21.15.72:6837","nonce":318582100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2380516950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2380516950},{"type":"v1","addr":"172.21.15.113:6835","nonce":2380516950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:54:32.882443+0000","modified":"2023-12-12T09:54:35.940697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5784":{"gid":5784,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6834/4181915308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":4181915308},{"type":"v1","addr":"172.21.15.113:6834","nonce":4181915308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-12T09:54:59.440 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 25 2023-12-12T09:54:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd4c006ae40 msgr2=0x7fd4c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd4c006ae40 0x7fd4c006d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0001eb0 tx=0x7fd4d0000f50 comp rx=0 tx=0).stop 2023-12-12T09:54:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4e015fc20 msgr2=0x7fd4e00b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4e015fc20 0x7fd4e00b23c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800f580 tx=0x7fd4c800fa40 comp rx=0 tx=0).stop 2023-12-12T09:54:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 shutdown_connections 2023-12-12T09:54:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd4c006ae40 0x7fd4c006d300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4e00b3f00 0x7fd4e00b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4e015fc20 0x7fd4e00b23c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 --2- 172.21.15.72:0/3993107483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4cc004d50 0x7fd4e013f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 >> 172.21.15.72:0/3993107483 conn(0x7fd4e00b0a90 msgr2=0x7fd4e00cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 shutdown_connections 2023-12-12T09:54:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.440+0000 7fd4e4eec640 1 -- 172.21.15.72:0/3993107483 wait complete. 2023-12-12T09:54:59.463 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:54:59.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.675+0000 7fdb1f91d640 1 Processor -- start 2023-12-12T09:54:59.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.675+0000 7fdb1f91d640 1 -- start start 2023-12-12T09:54:59.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1f91d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb18143e00 0x7fdb180b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1f91d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb180b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1f91d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb180b3bb0 con 0x7fdb1806c410 2023-12-12T09:54:59.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1f91d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb181441e0 con 0x7fdb18143e00 2023-12-12T09:54:59.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1f91d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18072680 con 0x7fdb18072180 2023-12-12T09:54:59.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1ceb8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb180b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1d6b9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb18143e00 0x7fdb180b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1ceb8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb180b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:53848/0 (socket says 172.21.15.72:53848) 2023-12-12T09:54:59.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.677+0000 7fdb1d6b9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb18143e00 0x7fdb180b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:57072/0 (socket says 172.21.15.72:57072) 2023-12-12T09:54:59.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.678+0000 7fdb1d6b9640 1 -- 172.21.15.72:0/3407525309 learned_addr learned my addr 172.21.15.72:0/3407525309 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:59.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.678+0000 7fdb17fff640 1 -- 172.21.15.72:0/3407525309 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188020878 0 0) 0x7fdb180b3bb0 con 0x7fdb1806c410 2023-12-12T09:54:59.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.678+0000 7fdb17fff640 1 -- 172.21.15.72:0/3407525309 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb00003520 con 0x7fdb1806c410 2023-12-12T09:54:59.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.678+0000 7fdb1ceb8640 1 -- 172.21.15.72:0/3407525309 >> v2:172.21.15.113:3300/0 conn(0x7fdb18143e00 msgr2=0x7fdb180b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:54:59.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.678+0000 7fdb1ceb8640 1 --2- 172.21.15.72:0/3407525309 >> v2:172.21.15.113:3300/0 conn(0x7fdb18143e00 0x7fdb180b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.678+0000 7fdb1ceb8640 1 -- 172.21.15.72:0/3407525309 >> v1:172.21.15.113:6789/0 conn(0x7fdb1806c410 legacy=0x7fdb18070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.678+0000 7fdb1ceb8640 1 -- 172.21.15.72:0/3407525309 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb180b1e30 con 0x7fdb18072180 2023-12-12T09:54:59.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.678+0000 7fdb1d6b9640 1 --2- 172.21.15.72:0/3407525309 >> v2:172.21.15.113:3300/0 conn(0x7fdb18143e00 0x7fdb180b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:54:59.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.680+0000 7fdb1ceb8640 1 --2- 172.21.15.72:0/3407525309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb180b4580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0097a0 tx=0x7fdb0c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78a773ae90d781b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:59.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.680+0000 7fdb17fff640 1 -- 172.21.15.72:0/3407525309 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c031830 con 0x7fdb18072180 2023-12-12T09:54:59.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.680+0000 7fdb17fff640 1 -- 172.21.15.72:0/3407525309 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c031990 con 0x7fdb18072180 2023-12-12T09:54:59.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.680+0000 7fdb17fff640 1 -- 172.21.15.72:0/3407525309 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c031d20 con 0x7fdb18072180 2023-12-12T09:54:59.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.681+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3407525309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 msgr2=0x7fdb180b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.681+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3407525309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb180b4580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0097a0 tx=0x7fdb0c02b8d0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.681+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3407525309 shutdown_connections 2023-12-12T09:54:59.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.681+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3407525309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb180b4580 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.681+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3407525309 >> v2:172.21.15.113:3300/0 conn(0x7fdb18143e00 0x7fdb180b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.681+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3407525309 >> 172.21.15.72:0/3407525309 conn(0x7fdb180b0a90 msgr2=0x7fdb180b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:59.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.681+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3407525309 shutdown_connections 2023-12-12T09:54:59.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.681+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3407525309 wait complete. 2023-12-12T09:54:59.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.683+0000 7fdb1f91d640 1 Processor -- start 2023-12-12T09:54:59.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.683+0000 7fdb1f91d640 1 -- start start 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1f91d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb1806c410 0x7fdb18142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1f91d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb18140360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1f91d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb18143e00 0x7fdb181408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1f91d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18139f00 con 0x7fdb1806c410 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1f91d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1813a150 con 0x7fdb18143e00 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1f91d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1813a430 con 0x7fdb18072180 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1d6b9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb1806c410 0x7fdb18142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1d6b9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb1806c410 0x7fdb18142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53270/0 (socket says 172.21.15.72:53270) 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.684+0000 7fdb1d6b9640 1 -- 172.21.15.72:0/3710549365 learned_addr learned my addr 172.21.15.72:0/3710549365 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1ceb8640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb18140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1deba640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb18143e00 0x7fdb181408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1d6b9640 1 -- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 msgr2=0x7fdb18140360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1d6b9640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb18140360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1d6b9640 1 -- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb18143e00 msgr2=0x7fdb181408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1d6b9640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb18143e00 0x7fdb181408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1d6b9640 1 -- 172.21.15.72:0/3710549365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08009b30 con 0x7fdb1806c410 2023-12-12T09:54:59.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1deba640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb18143e00 0x7fdb181408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:54:59.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1d6b9640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb1806c410 0x7fdb18142450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800efc0 tx=0x7fdb0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:59.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb15ffb640 1 -- 172.21.15.72:0/3710549365 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800ce60 con 0x7fdb1806c410 2023-12-12T09:54:59.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb15ffb640 1 -- 172.21.15.72:0/3710549365 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0800f040 con 0x7fdb1806c410 2023-12-12T09:54:59.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c009400 con 0x7fdb1806c410 2023-12-12T09:54:59.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.685+0000 7fdb15ffb640 1 -- 172.21.15.72:0/3710549365 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08014680 con 0x7fdb1806c410 2023-12-12T09:54:59.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.686+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1813a9f0 con 0x7fdb1806c410 2023-12-12T09:54:59.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.687+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb180745d0 con 0x7fdb1806c410 2023-12-12T09:54:59.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.687+0000 7fdb15ffb640 1 -- 172.21.15.72:0/3710549365 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb080147e0 con 0x7fdb1806c410 2023-12-12T09:54:59.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.688+0000 7fdb15ffb640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb0406ae40 0x7fdb0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:59.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.689+0000 7fdb15ffb640 1 -- 172.21.15.72:0/3710549365 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fdb0808a290 con 0x7fdb1806c410 2023-12-12T09:54:59.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.689+0000 7fdb1ceb8640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb0406ae40 0x7fdb0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:59.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.689+0000 7fdb1ceb8640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb0406ae40 0x7fdb0406d300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c036000 tx=0x7fdb0c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:59.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.693+0000 7fdb15ffb640 1 -- 172.21.15.72:0/3710549365 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb08055130 con 0x7fdb1806c410 2023-12-12T09:54:59.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.883+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb1806e020 con 0x7fdb1806c410 2023-12-12T09:54:59.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.886+0000 7fdb15ffb640 1 -- 172.21.15.72:0/3710549365 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16812 (secure 0 0 0) 0x7fdb08058750 con 0x7fdb1806c410 2023-12-12T09:54:59.888 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:54:59.888 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":53,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:54:56.470479+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:54:30.694215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-12T09:54:31.894291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.c","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:54:59.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb0406ae40 msgr2=0x7fdb0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb0406ae40 0x7fdb0406d300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c036000 tx=0x7fdb0c03c040 comp rx=0 tx=0).stop 2023-12-12T09:54:59.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb1806c410 msgr2=0x7fdb18142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb1806c410 0x7fdb18142450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800efc0 tx=0x7fdb0800c490 comp rx=0 tx=0).stop 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 shutdown_connections 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fdb0406ae40 0x7fdb0406d300 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb18143e00 0x7fdb181408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb18072180 0x7fdb18140360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 --2- 172.21.15.72:0/3710549365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb1806c410 0x7fdb18142450 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.888+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 >> 172.21.15.72:0/3710549365 conn(0x7fdb180b0a90 msgr2=0x7fdb18070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.889+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 shutdown_connections 2023-12-12T09:54:59.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:54:59.889+0000 7fdb1f91d640 1 -- 172.21.15.72:0/3710549365 wait complete. 2023-12-12T09:54:59.910 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-12T09:55:00.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.115+0000 7f501a368640 1 Processor -- start 2023-12-12T09:55:00.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.116+0000 7f501a368640 1 -- start start 2023-12-12T09:55:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.117+0000 7f501a368640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5014072180 0x7f50140b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.117+0000 7f501a368640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f501406c410 0x7f501415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.117+0000 7f501a368640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50140b3bb0 con 0x7f5014143e00 2023-12-12T09:55:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.117+0000 7f501a368640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50141441e0 con 0x7f5014072180 2023-12-12T09:55:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.117+0000 7f501a368640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5014072680 con 0x7f501406c410 2023-12-12T09:55:00.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.118+0000 7f5018905640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f501406c410 0x7f501415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.118+0000 7f500bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5014072180 0x7f50140b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.118+0000 7f5013fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5014143e00 0x7f50140b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.72:53104/0 (socket says 172.21.15.72:53104) 2023-12-12T09:55:00.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.118+0000 7f5013fff640 1 -- 172.21.15.72:0/1001113289 learned_addr learned my addr 172.21.15.72:0/1001113289 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:00.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.119+0000 7f500bfff640 1 -- 172.21.15.72:0/1001113289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f501406c410 msgr2=0x7f501415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:00.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.119+0000 7f500bfff640 1 --2- 172.21.15.72:0/1001113289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f501406c410 0x7f501415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.119+0000 7f500bfff640 1 -- 172.21.15.72:0/1001113289 >> v1:172.21.15.113:6789/0 conn(0x7f5014143e00 legacy=0x7f50140b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.119+0000 7f500bfff640 1 -- 172.21.15.72:0/1001113289 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140b1e30 con 0x7f5014072180 2023-12-12T09:55:00.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.119+0000 7f5018905640 1 --2- 172.21.15.72:0/1001113289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f501406c410 0x7f501415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:55:00.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.120+0000 7f500bfff640 1 --2- 172.21.15.72:0/1001113289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5014072180 0x7f50140b4750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0097a0 tx=0x7f4ffc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d482907a3dafdd62 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:00.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.121+0000 7f50137fe640 1 -- 172.21.15.72:0/1001113289 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc02d040 con 0x7f5014072180 2023-12-12T09:55:00.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.121+0000 7f50137fe640 1 -- 172.21.15.72:0/1001113289 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc031810 con 0x7f5014072180 2023-12-12T09:55:00.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.121+0000 7f50137fe640 1 -- 172.21.15.72:0/1001113289 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc031ba0 con 0x7f5014072180 2023-12-12T09:55:00.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.122+0000 7f501a368640 1 -- 172.21.15.72:0/1001113289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5014072180 msgr2=0x7f50140b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.122+0000 7f501a368640 1 --2- 172.21.15.72:0/1001113289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5014072180 0x7f50140b4750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0097a0 tx=0x7f4ffc02b8b0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.122+0000 7f501a368640 1 -- 172.21.15.72:0/1001113289 shutdown_connections 2023-12-12T09:55:00.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.122+0000 7f501a368640 1 --2- 172.21.15.72:0/1001113289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f501406c410 0x7f501415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.122+0000 7f501a368640 1 --2- 172.21.15.72:0/1001113289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5014072180 0x7f50140b4750 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.122+0000 7f501a368640 1 -- 172.21.15.72:0/1001113289 >> 172.21.15.72:0/1001113289 conn(0x7f50140b0a90 msgr2=0x7f50140b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:00.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.122+0000 7f501a368640 1 -- 172.21.15.72:0/1001113289 shutdown_connections 2023-12-12T09:55:00.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.122+0000 7f501a368640 1 -- 172.21.15.72:0/1001113289 wait complete. 2023-12-12T09:55:00.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.124+0000 7f501a368640 1 Processor -- start 2023-12-12T09:55:00.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.124+0000 7f501a368640 1 -- start start 2023-12-12T09:55:00.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f501a368640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501406c410 0x7f501413c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f501a368640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5014072180 0x7f5014136ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f501a368640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5014143e00 0x7f5014141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f501a368640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501415ae00 con 0x7f5014143e00 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f501a368640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501413b2e0 con 0x7f501406c410 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f501a368640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501413b5c0 con 0x7f5014072180 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f5018905640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5014143e00 0x7f5014141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f5018905640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5014143e00 0x7f5014141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53282/0 (socket says 172.21.15.72:53282) 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.125+0000 7f5018905640 1 -- 172.21.15.72:0/2406335221 learned_addr learned my addr 172.21.15.72:0/2406335221 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f5013fff640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501406c410 0x7f501413c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f500bfff640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5014072180 0x7f5014136ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f5018905640 1 -- 172.21.15.72:0/2406335221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5014072180 msgr2=0x7f5014136ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f5018905640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5014072180 0x7f5014136ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f5018905640 1 -- 172.21.15.72:0/2406335221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501406c410 msgr2=0x7f501413c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f5018905640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501406c410 0x7f501413c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f5018905640 1 -- 172.21.15.72:0/2406335221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c009a70 con 0x7f5014143e00 2023-12-12T09:55:00.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f5018905640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5014143e00 0x7f5014141e90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f500c00efc0 tx=0x7f500c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:00.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f50117fa640 1 -- 172.21.15.72:0/2406335221 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c009d70 con 0x7f5014143e00 2023-12-12T09:55:00.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f50117fa640 1 -- 172.21.15.72:0/2406335221 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c00f040 con 0x7f5014143e00 2023-12-12T09:55:00.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc009400 con 0x7f5014143e00 2023-12-12T09:55:00.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.126+0000 7f50117fa640 1 -- 172.21.15.72:0/2406335221 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c003990 con 0x7f5014143e00 2023-12-12T09:55:00.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.127+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501413bb80 con 0x7f5014143e00 2023-12-12T09:55:00.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.128+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50140745d0 con 0x7f5014143e00 2023-12-12T09:55:00.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.129+0000 7f50117fa640 1 -- 172.21.15.72:0/2406335221 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f500c0144a0 con 0x7f5014143e00 2023-12-12T09:55:00.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.129+0000 7f50117fa640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff406ae40 0x7f4ff406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.130+0000 7f50117fa640 1 -- 172.21.15.72:0/2406335221 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f500c08a3d0 con 0x7f5014143e00 2023-12-12T09:55:00.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.130+0000 7f5013fff640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff406ae40 0x7f4ff406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.130+0000 7f5013fff640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff406ae40 0x7f4ff406d300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5004000f60 tx=0x7f5004009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:00.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.134+0000 7f50117fa640 1 -- 172.21.15.72:0/2406335221 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f500c055300 con 0x7f5014143e00 2023-12-12T09:55:00.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.361+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f501406ddc0 con 0x7f5014143e00 2023-12-12T09:55:00.363 INFO:tasks.ceph.mon.a.smithi072.stderr:2023-12-12T09:55:00.362+0000 7f3fa4ad2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-12T09:55:00.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.365+0000 7f50117fa640 1 -- 172.21.15.72:0/2406335221 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f500c058920 con 0x7f5014143e00 2023-12-12T09:55:00.367 INFO:teuthology.orchestra.run.smithi072.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-12T09:55:00.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff406ae40 msgr2=0x7f4ff406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff406ae40 0x7f4ff406d300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5004000f60 tx=0x7f5004009040 comp rx=0 tx=0).stop 2023-12-12T09:55:00.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5014143e00 msgr2=0x7f5014141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5014143e00 0x7f5014141e90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f500c00efc0 tx=0x7f500c009f90 comp rx=0 tx=0).stop 2023-12-12T09:55:00.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 shutdown_connections 2023-12-12T09:55:00.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f4ff406ae40 0x7f4ff406d300 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5014143e00 0x7f5014141e90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5014072180 0x7f5014136ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 --2- 172.21.15.72:0/2406335221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f501406c410 0x7f501413c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.368+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 >> 172.21.15.72:0/2406335221 conn(0x7f50140b0a90 msgr2=0x7f50140cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:00.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.369+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 shutdown_connections 2023-12-12T09:55:00.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.369+0000 7f501a368640 1 -- 172.21.15.72:0/2406335221 wait complete. 2023-12-12T09:55:00.388 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-12T09:55:00.494 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-12-12T09:55:00.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.594+0000 7f51e1a1e640 1 Processor -- start 2023-12-12T09:55:00.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.595+0000 7f51e1a1e640 1 -- start start 2023-12-12T09:55:00.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.596+0000 7f51e1a1e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc0ce7e0 0x7f51dc15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.596+0000 7f51e1a1e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51dc06a460 0x7f51dc06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.596+0000 7f51e1a1e640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc071cd0 con 0x7f51dc15af50 2023-12-12T09:55:00.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.596+0000 7f51e1a1e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc06ae30 con 0x7f51dc06a460 2023-12-12T09:55:00.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.596+0000 7f51e1a1e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0717e0 con 0x7f51dc0ce7e0 2023-12-12T09:55:00.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.596+0000 7f51daffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f51dc15af50 0x7f51dc06c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36992/0 (socket says 172.21.15.72:36992) 2023-12-12T09:55:00.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.596+0000 7f51daffd640 1 -- 172.21.15.72:0/2576137897 learned_addr learned my addr 172.21.15.72:0/2576137897 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:00.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290783004 0 0) 0x7f51dc071cd0 con 0x7f51dc15af50 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51db7fe640 1 --2- 172.21.15.72:0/2576137897 >> v2:172.21.15.113:3300/0 conn(0x7f51dc06a460 0x7f51dc06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c8003520 con 0x7f51dc15af50 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51d3fff640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc0ce7e0 0x7f51dc15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3141134651 0 0) 0x7f51c8003520 con 0x7f51dc15af50 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc0ce7e0 msgr2=0x7f51dc15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc0ce7e0 0x7f51dc15ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 >> v2:172.21.15.113:3300/0 conn(0x7f51dc06a460 msgr2=0x7f51dc06a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 --2- 172.21.15.72:0/2576137897 >> v2:172.21.15.113:3300/0 conn(0x7f51dc06a460 0x7f51dc06a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0719f0 con 0x7f51dc15af50 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f51cc001670 con 0x7f51dc15af50 2023-12-12T09:55:00.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 >> v1:172.21.15.72:6789/0 conn(0x7f51dc15af50 legacy=0x7f51dc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.597+0000 7f51da7fc640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc0ce7e0 0x7f51c8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51da7fc640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51da7fc640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c8004d00 0x7f51c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc06ae30 con 0x7f51c8003f40 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0717e0 con 0x7f51c8004d00 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c800bb60 con 0x7f51dc0ce7e0 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51d3fff640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51daffd640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc0ce7e0 0x7f51c8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51db7fe640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c8004d00 0x7f51c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51d3fff640 1 -- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc0ce7e0 msgr2=0x7f51c8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51d3fff640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc0ce7e0 0x7f51c8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51d3fff640 1 -- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c8004d00 msgr2=0x7f51c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:00.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51d3fff640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c8004d00 0x7f51c8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.598+0000 7f51d3fff640 1 -- 172.21.15.72:0/2576137897 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8003520 con 0x7f51c8003f40 2023-12-12T09:55:00.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.599+0000 7f51d3fff640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51c80047c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f51c400ba30 tx=0x7f51c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7712794f5f6c81dd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:00.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4032640 con 0x7f51c8003f40 2023-12-12T09:55:00.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c40327a0 con 0x7f51c8003f40 2023-12-12T09:55:00.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51da7fc640 1 -- 172.21.15.72:0/2576137897 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4032b30 con 0x7f51c8003f40 2023-12-12T09:55:00.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2576137897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 msgr2=0x7f51c80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51e1a1e640 1 --2- 172.21.15.72:0/2576137897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51c80047c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f51c400ba30 tx=0x7f51c80057f0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51db7fe640 1 -- 172.21.15.72:0/2576137897 reap_dead start 2023-12-12T09:55:00.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2576137897 shutdown_connections 2023-12-12T09:55:00.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2576137897 >> 172.21.15.72:0/2576137897 conn(0x7f51dc0cee70 msgr2=0x7f51dc0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:00.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2576137897 shutdown_connections 2023-12-12T09:55:00.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.600+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2576137897 wait complete. 2023-12-12T09:55:00.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.602+0000 7f51e1a1e640 1 Processor -- start 2023-12-12T09:55:00.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.602+0000 7f51e1a1e640 1 -- start start 2023-12-12T09:55:00.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51e1a1e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51dc0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51e1a1e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51dc151250 0x7f51dc09bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51e1a1e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc09c100 0x7f51dc0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51e1a1e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc09c580 con 0x7f51c8003f40 2023-12-12T09:55:00.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51e1a1e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0c8f80 con 0x7f51dc151250 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51e1a1e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0c7bf0 con 0x7f51dc09c100 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51daffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51dc0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51daffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51dc0c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53296/0 (socket says 172.21.15.72:53296) 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.603+0000 7f51daffd640 1 -- 172.21.15.72:0/2893606422 learned_addr learned my addr 172.21.15.72:0/2893606422 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51db7fe640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc09c100 0x7f51dc0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51daffd640 1 -- 172.21.15.72:0/2893606422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc09c100 msgr2=0x7f51dc0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51d3fff640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51dc151250 0x7f51dc09bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51daffd640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc09c100 0x7f51dc0c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51daffd640 1 -- 172.21.15.72:0/2893606422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51dc151250 msgr2=0x7f51dc09bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51daffd640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51dc151250 0x7f51dc09bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51daffd640 1 -- 172.21.15.72:0/2893606422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0c8370 con 0x7f51c8003f40 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51daffd640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51dc0c76b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00e700 tx=0x7f51cc00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:00.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51d8ff9640 1 -- 172.21.15.72:0/2893606422 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00c6c0 con 0x7f51c8003f40 2023-12-12T09:55:00.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.604+0000 7f51d8ff9640 1 -- 172.21.15.72:0/2893606422 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc00c820 con 0x7f51c8003f40 2023-12-12T09:55:00.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.605+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc0c8650 con 0x7f51c8003f40 2023-12-12T09:55:00.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.605+0000 7f51d8ff9640 1 -- 172.21.15.72:0/2893606422 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00cbd0 con 0x7f51c8003f40 2023-12-12T09:55:00.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.605+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc0c8c10 con 0x7f51c8003f40 2023-12-12T09:55:00.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.606+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc074450 con 0x7f51c8003f40 2023-12-12T09:55:00.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.607+0000 7f51d8ff9640 1 -- 172.21.15.72:0/2893606422 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51cc00cd30 con 0x7f51c8003f40 2023-12-12T09:55:00.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.607+0000 7f51d8ff9640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f51bc06ae40 0x7f51bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:00.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.608+0000 7f51d8ff9640 1 -- 172.21.15.72:0/2893606422 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f51cc089690 con 0x7f51c8003f40 2023-12-12T09:55:00.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.608+0000 7f51d3fff640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f51bc06ae40 0x7f51bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:00.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.608+0000 7f51d3fff640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f51bc06ae40 0x7f51bc06d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f51c400ba00 tx=0x7f51c4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:00.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.612+0000 7f51d8ff9640 1 -- 172.21.15.72:0/2893606422 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51cc0544d0 con 0x7f51c8003f40 2023-12-12T09:55:00.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:00.843+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f51dc0ce1d0 con 0x7f51c8003f40 2023-12-12T09:55:01.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.517+0000 7f51d8ff9640 1 -- 172.21.15.72:0/2893606422 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f51cc057af0 con 0x7f51c8003f40 2023-12-12T09:55:01.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f51bc06ae40 msgr2=0x7f51bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:01.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f51bc06ae40 0x7f51bc06d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f51c400ba00 tx=0x7f51c4030000 comp rx=0 tx=0).stop 2023-12-12T09:55:01.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 msgr2=0x7f51dc0c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51dc0c76b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00e700 tx=0x7f51cc00ebc0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 shutdown_connections 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f51bc06ae40 0x7f51bc06d300 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51dc09c100 0x7f51dc0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51dc151250 0x7f51dc09bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 --2- 172.21.15.72:0/2893606422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c8003f40 0x7f51dc0c76b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.520+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 >> 172.21.15.72:0/2893606422 conn(0x7f51dc0cee70 msgr2=0x7f51dc070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.521+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 shutdown_connections 2023-12-12T09:55:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.521+0000 7f51e1a1e640 1 -- 172.21.15.72:0/2893606422 wait complete. 2023-12-12T09:55:01.542 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-12T09:55:01.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.760+0000 7f930095c640 1 Processor -- start 2023-12-12T09:55:01.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.761+0000 7f930095c640 1 -- start start 2023-12-12T09:55:01.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.762+0000 7f930095c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f92fc0d15b0 0x7f92fc0d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:01.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.762+0000 7f930095c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc15d4a0 0x7f92fc0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:01.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.762+0000 7f930095c640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92fc0ce5d0 con 0x7f92fc0d1990 2023-12-12T09:55:01.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.762+0000 7f930095c640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92fc15d8d0 con 0x7f92fc0d15b0 2023-12-12T09:55:01.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.762+0000 7f930095c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0ce870 con 0x7f92fc15d4a0 2023-12-12T09:55:01.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.762+0000 7f92fb7fe640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f92fc0d15b0 0x7f92fc0d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:01.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.762+0000 7f92faffd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f92fc0d1990 0x7f92fc15cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36994/0 (socket says 172.21.15.72:36994) 2023-12-12T09:55:01.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.762+0000 7f92faffd640 1 -- 172.21.15.72:0/2583363939 learned_addr learned my addr 172.21.15.72:0/2583363939 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:01.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.763+0000 7f92fbfff640 1 --2- 172.21.15.72:0/2583363939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc15d4a0 0x7f92fc0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:01.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.763+0000 7f92fb7fe640 1 -- 172.21.15.72:0/2583363939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc15d4a0 msgr2=0x7f92fc0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:01.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.763+0000 7f92fb7fe640 1 --2- 172.21.15.72:0/2583363939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc15d4a0 0x7f92fc0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.763+0000 7f92fb7fe640 1 -- 172.21.15.72:0/2583363939 >> v1:172.21.15.72:6789/0 conn(0x7f92fc0d1990 legacy=0x7f92fc15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:01.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.763+0000 7f92fb7fe640 1 -- 172.21.15.72:0/2583363939 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc0ceb50 con 0x7f92fc0d15b0 2023-12-12T09:55:01.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.763+0000 7f92fbfff640 1 --2- 172.21.15.72:0/2583363939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc15d4a0 0x7f92fc0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:01.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.763+0000 7f92fa7fc640 1 -- 172.21.15.72:0/2583363939 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885424306 0 0) 0x7f92fc0ce5d0 con 0x7f92fc0d1990 2023-12-12T09:55:01.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.764+0000 7f92fb7fe640 1 --2- 172.21.15.72:0/2583363939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d15b0 0x7f92fc0d3a30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0097a0 tx=0x7f92ec02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971a8e534484452d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:01.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.764+0000 7f92fa7fc640 1 -- 172.21.15.72:0/2583363939 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec031780 con 0x7f92fc0d15b0 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.764+0000 7f92fa7fc640 1 -- 172.21.15.72:0/2583363939 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec0318e0 con 0x7f92fc0d15b0 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.764+0000 7f92fa7fc640 1 -- 172.21.15.72:0/2583363939 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec031c70 con 0x7f92fc0d15b0 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.765+0000 7f930095c640 1 -- 172.21.15.72:0/2583363939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d15b0 msgr2=0x7f92fc0d3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.765+0000 7f930095c640 1 --2- 172.21.15.72:0/2583363939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d15b0 0x7f92fc0d3a30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0097a0 tx=0x7f92ec02b820 comp rx=0 tx=0).stop 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.765+0000 7f930095c640 1 -- 172.21.15.72:0/2583363939 shutdown_connections 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.765+0000 7f930095c640 1 --2- 172.21.15.72:0/2583363939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc15d4a0 0x7f92fc0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.765+0000 7f930095c640 1 --2- 172.21.15.72:0/2583363939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d15b0 0x7f92fc0d3a30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.765+0000 7f930095c640 1 -- 172.21.15.72:0/2583363939 >> 172.21.15.72:0/2583363939 conn(0x7f92fc0d2f10 msgr2=0x7f92fc0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.765+0000 7f930095c640 1 -- 172.21.15.72:0/2583363939 shutdown_connections 2023-12-12T09:55:01.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.765+0000 7f930095c640 1 -- 172.21.15.72:0/2583363939 wait complete. 2023-12-12T09:55:01.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.767+0000 7f930095c640 1 Processor -- start 2023-12-12T09:55:01.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.767+0000 7f930095c640 1 -- start start 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f930095c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc0d15b0 0x7f92fc0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f930095c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d1990 0x7f92fc1f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f930095c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92fc15d4a0 0x7f92fc1f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f930095c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0cb1d0 con 0x7f92fc0d1990 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f930095c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc1f37e0 con 0x7f92fc15d4a0 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f930095c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc1f2670 con 0x7f92fc0d15b0 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f92faffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d1990 0x7f92fc1f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f92faffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d1990 0x7f92fc1f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53316/0 (socket says 172.21.15.72:53316) 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.768+0000 7f92faffd640 1 -- 172.21.15.72:0/2498525550 learned_addr learned my addr 172.21.15.72:0/2498525550 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92fbfff640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92fc15d4a0 0x7f92fc1f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92fb7fe640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc0d15b0 0x7f92fc0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:01.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92faffd640 1 -- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc0d15b0 msgr2=0x7f92fc0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92faffd640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc0d15b0 0x7f92fc0c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92faffd640 1 -- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92fc15d4a0 msgr2=0x7f92fc1f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92faffd640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92fc15d4a0 0x7f92fc1f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92faffd640 1 -- 172.21.15.72:0/2498525550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec009400 con 0x7f92fc0d1990 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92fbfff640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92fc15d4a0 0x7f92fc1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92faffd640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d1990 0x7f92fc1f32a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c890 tx=0x7f92e400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92f3fff640 1 -- 172.21.15.72:0/2498525550 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e400a640 con 0x7f92fc0d1990 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92f3fff640 1 -- 172.21.15.72:0/2498525550 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e400a7c0 con 0x7f92fc0d1990 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc0ca5c0 con 0x7f92fc0d1990 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.769+0000 7f92f3fff640 1 -- 172.21.15.72:0/2498525550 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e400ab70 con 0x7f92fc0d1990 2023-12-12T09:55:01.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.770+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc0cab80 con 0x7f92fc0d1990 2023-12-12T09:55:01.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.771+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92fc0cb380 con 0x7f92fc0d1990 2023-12-12T09:55:01.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.772+0000 7f92f3fff640 1 -- 172.21.15.72:0/2498525550 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92e400acd0 con 0x7f92fc0d1990 2023-12-12T09:55:01.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.773+0000 7f92f3fff640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f92e006ae40 0x7f92e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:01.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.773+0000 7f92f3fff640 1 -- 172.21.15.72:0/2498525550 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f92e40891b0 con 0x7f92fc0d1990 2023-12-12T09:55:01.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.773+0000 7f92fb7fe640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f92e006ae40 0x7f92e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:01.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.774+0000 7f92fb7fe640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f92e006ae40 0x7f92e006d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f92ec036000 tx=0x7f92ec03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:01.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.777+0000 7f92f3fff640 1 -- 172.21.15.72:0/2498525550 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92fc0cb380 con 0x7f92fc0d1990 2023-12-12T09:55:01.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:01.967+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92fc074450 con 0x7f92fc0d1990 2023-12-12T09:55:02.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.527+0000 7f92f3fff640 1 -- 172.21.15.72:0/2498525550 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f92e4053ff0 con 0x7f92fc0d1990 2023-12-12T09:55:02.529 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' removed 2023-12-12T09:55:02.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.530+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f92e006ae40 msgr2=0x7f92e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:02.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.530+0000 7f930095c640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f92e006ae40 0x7f92e006d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f92ec036000 tx=0x7f92ec03d040 comp rx=0 tx=0).stop 2023-12-12T09:55:02.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.530+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d1990 msgr2=0x7f92fc1f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:02.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d1990 0x7f92fc1f32a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c890 tx=0x7f92e400cd50 comp rx=0 tx=0).stop 2023-12-12T09:55:02.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 shutdown_connections 2023-12-12T09:55:02.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f92e006ae40 0x7f92e006d300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92fc15d4a0 0x7f92fc1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92fc0d1990 0x7f92fc1f32a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 --2- 172.21.15.72:0/2498525550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92fc0d15b0 0x7f92fc0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 >> 172.21.15.72:0/2498525550 conn(0x7f92fc0d2f10 msgr2=0x7f92fc0cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:02.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 shutdown_connections 2023-12-12T09:55:02.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.531+0000 7f930095c640 1 -- 172.21.15.72:0/2498525550 wait complete. 2023-12-12T09:55:02.556 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-12T09:55:02.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.759+0000 7fecabcc6640 1 Processor -- start 2023-12-12T09:55:02.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.759+0000 7fecabcc6640 1 -- start start 2023-12-12T09:55:02.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.760+0000 7fecabcc6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feca41448a0 0x7feca406c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7fecabcc6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 0x7feca415f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7fecabcc6640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca40b3c50 con 0x7feca40724a0 2023-12-12T09:55:02.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7fecabcc6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca40b3960 con 0x7feca41448a0 2023-12-12T09:55:02.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7fecabcc6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca415f550 con 0x7feca415ec10 2023-12-12T09:55:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7feca9261640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7feca40724a0 0x7feca40b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36996/0 (socket says 172.21.15.72:36996) 2023-12-12T09:55:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7feca9261640 1 -- 172.21.15.72:0/1188273671 learned_addr learned my addr 172.21.15.72:0/1188273671 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175502528 0 0) 0x7feca40b3c50 con 0x7feca40724a0 2023-12-12T09:55:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec90003520 con 0x7feca40724a0 2023-12-12T09:55:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7feca9a62640 1 --2- 172.21.15.72:0/1188273671 >> v2:172.21.15.113:3300/0 conn(0x7feca41448a0 0x7feca406c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.761+0000 7fecaa263640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 0x7feca415f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874006856 0 0) 0x7fec90003520 con 0x7feca40724a0 2023-12-12T09:55:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 msgr2=0x7feca415f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 0x7feca415f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 >> v2:172.21.15.113:3300/0 conn(0x7feca41448a0 msgr2=0x7feca406c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 --2- 172.21.15.72:0/1188273671 >> v2:172.21.15.113:3300/0 conn(0x7feca41448a0 0x7feca406c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca4071140 con 0x7feca40724a0 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fec94001670 con 0x7feca40724a0 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7fecaa263640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 0x7feca415f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 >> v1:172.21.15.72:6789/0 conn(0x7feca40724a0 legacy=0x7feca40b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 0x7fec90003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90003f00 0x7fec900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec90004d00 0x7fec90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca40b3960 con 0x7fec90004d00 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca415f550 con 0x7fec90003f00 2023-12-12T09:55:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9000ba00 con 0x7feca415ec10 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.762+0000 7fecaa263640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec90004d00 0x7fec90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.763+0000 7feca9a62640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 0x7fec90003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.763+0000 7feca9261640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90003f00 0x7fec900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.763+0000 7fecaa263640 1 -- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 msgr2=0x7fec90003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.763+0000 7fecaa263640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca415ec10 0x7fec90003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.763+0000 7fecaa263640 1 -- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90003f00 msgr2=0x7fec900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.763+0000 7fecaa263640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90003f00 0x7fec900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.763+0000 7fecaa263640 1 -- 172.21.15.72:0/1188273671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90003520 con 0x7fec90004d00 2023-12-12T09:55:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.763+0000 7feca9261640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90003f00 0x7fec900047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.764+0000 7fecaa263640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec90004d00 0x7fec90005180 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0073e0 tx=0x7fec900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d6007f5a1fbc49 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.764+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c02cd30 con 0x7fec90004d00 2023-12-12T09:55:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.764+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c02ce90 con 0x7fec90004d00 2023-12-12T09:55:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.764+0000 7feca8a60640 1 -- 172.21.15.72:0/1188273671 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c034640 con 0x7fec90004d00 2023-12-12T09:55:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.765+0000 7fecabcc6640 1 -- 172.21.15.72:0/1188273671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec90004d00 msgr2=0x7fec90005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:02.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.765+0000 7fecabcc6640 1 --2- 172.21.15.72:0/1188273671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec90004d00 0x7fec90005180 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0073e0 tx=0x7fec900057f0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.765+0000 7fecaa263640 1 -- 172.21.15.72:0/1188273671 reap_dead start 2023-12-12T09:55:02.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.765+0000 7fecabcc6640 1 -- 172.21.15.72:0/1188273671 shutdown_connections 2023-12-12T09:55:02.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.765+0000 7fecabcc6640 1 -- 172.21.15.72:0/1188273671 >> 172.21.15.72:0/1188273671 conn(0x7feca40b0a90 msgr2=0x7feca40b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:02.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.765+0000 7fecabcc6640 1 -- 172.21.15.72:0/1188273671 shutdown_connections 2023-12-12T09:55:02.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.765+0000 7fecabcc6640 1 -- 172.21.15.72:0/1188273671 wait complete. 2023-12-12T09:55:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.767+0000 7fecabcc6640 1 Processor -- start 2023-12-12T09:55:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.767+0000 7fecabcc6640 1 -- start start 2023-12-12T09:55:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecabcc6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90004d00 0x7feca41419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecabcc6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca413eef0 0x7feca413c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecabcc6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca413c9f0 0x7feca41f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecabcc6640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca41f6250 con 0x7feca413c9f0 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecabcc6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca41f64a0 con 0x7fec90004d00 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecabcc6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca41f6780 con 0x7feca413eef0 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecaa263640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca413c9f0 0x7feca41f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecaa263640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca413c9f0 0x7feca41f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53336/0 (socket says 172.21.15.72:53336) 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.768+0000 7fecaa263640 1 -- 172.21.15.72:0/2422033533 learned_addr learned my addr 172.21.15.72:0/2422033533 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7feca9261640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca413eef0 0x7feca413c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7feca9a62640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90004d00 0x7feca41419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fecaa263640 1 -- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca413eef0 msgr2=0x7feca413c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fecaa263640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca413eef0 0x7feca413c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fecaa263640 1 -- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90004d00 msgr2=0x7feca41419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fecaa263640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90004d00 0x7feca41419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:02.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fecaa263640 1 -- 172.21.15.72:0/2422033533 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca41f6a60 con 0x7feca413c9f0 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7feca9261640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca413eef0 0x7feca413c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fecaa263640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca413c9f0 0x7feca41f2f90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fec9c037040 tx=0x7fec9c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fec9a7fc640 1 -- 172.21.15.72:0/2422033533 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c02cd30 con 0x7feca413c9f0 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fec9a7fc640 1 -- 172.21.15.72:0/2422033533 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c003990 con 0x7feca413c9f0 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fec9a7fc640 1 -- 172.21.15.72:0/2422033533 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c003d20 con 0x7feca413c9f0 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.769+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca41f6d40 con 0x7feca413c9f0 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.770+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca4141ee0 con 0x7feca413c9f0 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.770+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca4074670 con 0x7feca413c9f0 2023-12-12T09:55:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.771+0000 7fec9a7fc640 1 -- 172.21.15.72:0/2422033533 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec9c03b050 con 0x7feca413c9f0 2023-12-12T09:55:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.772+0000 7fec9a7fc640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fec8806ae40 0x7fec8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.772+0000 7fec9a7fc640 1 -- 172.21.15.72:0/2422033533 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5735+0+0 (secure 0 0 0) 0x7fec9c0ab540 con 0x7feca413c9f0 2023-12-12T09:55:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.773+0000 7feca9a62640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fec8806ae40 0x7fec8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:02.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.773+0000 7feca9a62640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fec8806ae40 0x7fec8806d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feca00099d0 tx=0x7feca0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:02.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.777+0000 7fec9a7fc640 1 -- 172.21.15.72:0/2422033533 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec9c076670 con 0x7feca413c9f0 2023-12-12T09:55:02.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:02.967+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feca41f6250 con 0x7feca413c9f0 2023-12-12T09:55:03.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.540+0000 7fec9a7fc640 1 -- 172.21.15.72:0/2422033533 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7fec9c079c90 con 0x7feca413c9f0 2023-12-12T09:55:03.542 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' removed 2023-12-12T09:55:03.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.543+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fec8806ae40 msgr2=0x7fec8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.543+0000 7fecabcc6640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fec8806ae40 0x7fec8806d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feca00099d0 tx=0x7feca0008040 comp rx=0 tx=0).stop 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.543+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca413c9f0 msgr2=0x7feca41f2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.543+0000 7fecabcc6640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca413c9f0 0x7feca41f2f90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fec9c037040 tx=0x7fec9c004ab0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.544+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 shutdown_connections 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.544+0000 7fecabcc6640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fec8806ae40 0x7fec8806d300 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.544+0000 7fecabcc6640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca413c9f0 0x7feca41f2f90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.544+0000 7fecabcc6640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feca413eef0 0x7feca413c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.544+0000 7fecabcc6640 1 --2- 172.21.15.72:0/2422033533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec90004d00 0x7feca41419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.544+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 >> 172.21.15.72:0/2422033533 conn(0x7feca40b0a90 msgr2=0x7feca406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.544+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 shutdown_connections 2023-12-12T09:55:03.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.544+0000 7fecabcc6640 1 -- 172.21.15.72:0/2422033533 wait complete. 2023-12-12T09:55:03.567 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_fragment_size_max 2023-12-12T09:55:03.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.775+0000 7f0e39cf3640 1 Processor -- start 2023-12-12T09:55:03.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.776+0000 7f0e39cf3640 1 -- start start 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.777+0000 7f0e39cf3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e340c2390 0x7f0e340ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.777+0000 7f0e39cf3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0e340ca4f0 0x7f0e341652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.777+0000 7f0e39cf3640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e340ca900 con 0x7f0e34071fb0 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.777+0000 7f0e39cf3640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e340caf80 con 0x7f0e340ca4f0 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.777+0000 7f0e39cf3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3406a5e0 con 0x7f0e340c2390 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.777+0000 7f0e33fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0e340ca4f0 0x7f0e341652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.777+0000 7f0e337fe640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f0e34071fb0 0x7f0e340c1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:37004/0 (socket says 172.21.15.72:37004) 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.777+0000 7f0e337fe640 1 -- 172.21.15.72:0/3447007694 learned_addr learned my addr 172.21.15.72:0/3447007694 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.778+0000 7f0e2affd640 1 --2- 172.21.15.72:0/3447007694 >> v2:172.21.15.113:3300/0 conn(0x7f0e340c2390 0x7f0e340ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.778+0000 7f0e33fff640 1 -- 172.21.15.72:0/3447007694 >> v2:172.21.15.113:3300/0 conn(0x7f0e340c2390 msgr2=0x7f0e340ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:03.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.778+0000 7f0e33fff640 1 --2- 172.21.15.72:0/3447007694 >> v2:172.21.15.113:3300/0 conn(0x7f0e340c2390 0x7f0e340ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.778+0000 7f0e33fff640 1 -- 172.21.15.72:0/3447007694 >> v1:172.21.15.72:6789/0 conn(0x7f0e34071fb0 legacy=0x7f0e340c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:03.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.778+0000 7f0e33fff640 1 -- 172.21.15.72:0/3447007694 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3406a8c0 con 0x7f0e340ca4f0 2023-12-12T09:55:03.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.779+0000 7f0e33fff640 1 --2- 172.21.15.72:0/3447007694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340ca4f0 0x7f0e341652e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0e34070480 tx=0x7f0e2c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cab106af2db37a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:03.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.779+0000 7f0e32ffd640 1 -- 172.21.15.72:0/3447007694 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c02cc70 con 0x7f0e340ca4f0 2023-12-12T09:55:03.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.779+0000 7f0e32ffd640 1 -- 172.21.15.72:0/3447007694 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c02cdd0 con 0x7f0e340ca4f0 2023-12-12T09:55:03.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.779+0000 7f0e32ffd640 1 -- 172.21.15.72:0/3447007694 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c034660 con 0x7f0e340ca4f0 2023-12-12T09:55:03.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.780+0000 7f0e39cf3640 1 -- 172.21.15.72:0/3447007694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340ca4f0 msgr2=0x7f0e341652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:03.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.780+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/3447007694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340ca4f0 0x7f0e341652e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0e34070480 tx=0x7f0e2c02bec0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.780+0000 7f0e39cf3640 1 -- 172.21.15.72:0/3447007694 shutdown_connections 2023-12-12T09:55:03.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.780+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/3447007694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340ca4f0 0x7f0e341652e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.780+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/3447007694 >> v2:172.21.15.113:3300/0 conn(0x7f0e340c2390 0x7f0e340ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.780+0000 7f0e39cf3640 1 -- 172.21.15.72:0/3447007694 >> 172.21.15.72:0/3447007694 conn(0x7f0e340cee70 msgr2=0x7f0e340cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:03.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.780+0000 7f0e39cf3640 1 -- 172.21.15.72:0/3447007694 shutdown_connections 2023-12-12T09:55:03.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.780+0000 7f0e39cf3640 1 -- 172.21.15.72:0/3447007694 wait complete. 2023-12-12T09:55:03.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.782+0000 7f0e39cf3640 1 Processor -- start 2023-12-12T09:55:03.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.782+0000 7f0e39cf3640 1 -- start start 2023-12-12T09:55:03.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e39cf3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e34071fb0 0x7f0e340c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e39cf3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340c2390 0x7f0e34158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e39cf3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e340ca4f0 0x7f0e340c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e39cf3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3415b2c0 con 0x7f0e340c2390 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e39cf3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340c5f00 con 0x7f0e34071fb0 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e39cf3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340c61e0 con 0x7f0e340ca4f0 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e2affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340c2390 0x7f0e34158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e2affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340c2390 0x7f0e34158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53366/0 (socket says 172.21.15.72:53366) 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.783+0000 7f0e2affd640 1 -- 172.21.15.72:0/1406932486 learned_addr learned my addr 172.21.15.72:0/1406932486 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e337fe640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e34071fb0 0x7f0e340c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e33fff640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e340ca4f0 0x7f0e340c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e2affd640 1 -- 172.21.15.72:0/1406932486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e340ca4f0 msgr2=0x7f0e340c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e2affd640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e340ca4f0 0x7f0e340c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e2affd640 1 -- 172.21.15.72:0/1406932486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e34071fb0 msgr2=0x7f0e340c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e2affd640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e34071fb0 0x7f0e340c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e2affd640 1 -- 172.21.15.72:0/1406932486 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c009a70 con 0x7f0e340c2390 2023-12-12T09:55:03.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e2affd640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340c2390 0x7f0e34158fa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00d7d0 tx=0x7f0e1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:03.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e30ff9640 1 -- 172.21.15.72:0/1406932486 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c018070 con 0x7f0e340c2390 2023-12-12T09:55:03.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e30ff9640 1 -- 172.21.15.72:0/1406932486 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c004020 con 0x7f0e340c2390 2023-12-12T09:55:03.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e30ff9640 1 -- 172.21.15.72:0/1406932486 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0041b0 con 0x7f0e340c2390 2023-12-12T09:55:03.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.784+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e340c64c0 con 0x7f0e340c2390 2023-12-12T09:55:03.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.785+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e341597e0 con 0x7f0e340c2390 2023-12-12T09:55:03.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.786+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e34074450 con 0x7f0e340c2390 2023-12-12T09:55:03.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.787+0000 7f0e30ff9640 1 -- 172.21.15.72:0/1406932486 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e1c004310 con 0x7f0e340c2390 2023-12-12T09:55:03.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.787+0000 7f0e30ff9640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0e1406ae70 0x7f0e1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:03.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.788+0000 7f0e30ff9640 1 -- 172.21.15.72:0/1406932486 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0e1c089410 con 0x7f0e340c2390 2023-12-12T09:55:03.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.788+0000 7f0e337fe640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0e1406ae70 0x7f0e1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:03.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.788+0000 7f0e337fe640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0e1406ae70 0x7f0e1406d330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0e24005880 tx=0x7f0e24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:03.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.792+0000 7f0e30ff9640 1 -- 172.21.15.72:0/1406932486 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e1c054550 con 0x7f0e340c2390 2023-12-12T09:55:03.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.985+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_fragment_size_max"} v 0) v1 -- 0x7f0e340ca2d0 con 0x7f0e340c2390 2023-12-12T09:55:03.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.991+0000 7f0e30ff9640 1 -- 172.21.15.72:0/1406932486 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_fragment_size_max"}]=0 v14) v1 ==== 108+0+0 (secure 0 0 0) 0x7f0e1c057b70 con 0x7f0e340c2390 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0e1406ae70 msgr2=0x7f0e1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0e1406ae70 0x7f0e1406d330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0e24005880 tx=0x7f0e24009040 comp rx=0 tx=0).stop 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340c2390 msgr2=0x7f0e34158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340c2390 0x7f0e34158fa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00d7d0 tx=0x7f0e1c00dc90 comp rx=0 tx=0).stop 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 shutdown_connections 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f0e1406ae70 0x7f0e1406d330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e340ca4f0 0x7f0e340c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e340c2390 0x7f0e34158fa0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 --2- 172.21.15.72:0/1406932486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e34071fb0 0x7f0e340c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:03.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 >> 172.21.15.72:0/1406932486 conn(0x7f0e340cee70 msgr2=0x7f0e3406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:03.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 shutdown_connections 2023-12-12T09:55:03.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:03.994+0000 7f0e39cf3640 1 -- 172.21.15.72:0/1406932486 wait complete. 2023-12-12T09:55:04.013 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-12T09:55:04.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.220+0000 7fe15ab83640 1 Processor -- start 2023-12-12T09:55:04.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.221+0000 7fe15ab83640 1 -- start start 2023-12-12T09:55:04.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.222+0000 7fe15ab83640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1540cfe70 0x7fe154168530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.222+0000 7fe15ab83640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540c6b80 con 0x7fe1540b3350 2023-12-12T09:55:04.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.222+0000 7fe15ab83640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540d3bf0 con 0x7fe1540b3730 2023-12-12T09:55:04.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.222+0000 7fe15ab83640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1540d3ed0 con 0x7fe1540cfe70 2023-12-12T09:55:04.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.222+0000 7fe159b81640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fe1540b3350 0x7fe1540b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:37018/0 (socket says 172.21.15.72:37018) 2023-12-12T09:55:04.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe159b81640 1 -- 172.21.15.72:0/3594417434 learned_addr learned my addr 172.21.15.72:0/3594417434 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:04.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe15a382640 1 --2- 172.21.15.72:0/3594417434 >> v2:172.21.15.113:3300/0 conn(0x7fe1540cfe70 0x7fe154168530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069253048 0 0) 0x7fe1540c6b80 con 0x7fe1540b3350 2023-12-12T09:55:04.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe13c003520 con 0x7fe1540b3350 2023-12-12T09:55:04.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1469574654 0 0) 0x7fe13c003520 con 0x7fe1540b3350 2023-12-12T09:55:04.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 >> v2:172.21.15.113:3300/0 conn(0x7fe1540cfe70 msgr2=0x7fe154168530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:04.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 --2- 172.21.15.72:0/3594417434 >> v2:172.21.15.113:3300/0 conn(0x7fe1540cfe70 0x7fe154168530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 >> v1:172.21.15.113:6789/0 conn(0x7fe1540b3730 legacy=0x7fe1540cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540d18c0 con 0x7fe1540b3350 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fe1440019f0 con 0x7fe1540b3350 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 >> v1:172.21.15.72:6789/0 conn(0x7fe1540b3350 legacy=0x7fe1540b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.223+0000 7fe158b7f640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe13c003970 0x7fe13c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe158b7f640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe13c0047c0 0x7fe13c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe158b7f640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c005180 0x7fe13c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540d3ed0 con 0x7fe13c003970 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c00c020 con 0x7fe13c005180 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c00c1e0 con 0x7fe13c0047c0 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe159b81640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe13c003970 0x7fe13c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe15a382640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c005180 0x7fe13c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe159b81640 1 -- 172.21.15.72:0/3594417434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe13c0047c0 msgr2=0x7fe13c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe159b81640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe13c0047c0 0x7fe13c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe159b81640 1 -- 172.21.15.72:0/3594417434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c005180 msgr2=0x7fe13c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe159b81640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe13c005180 0x7fe13c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe159b81640 1 -- 172.21.15.72:0/3594417434 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c003520 con 0x7fe13c003970 2023-12-12T09:55:04.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.224+0000 7fe15a382640 1 -- 172.21.15.72:0/3594417434 reap_dead start 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.226+0000 7fe159b81640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe13c003970 0x7fe13c003d90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe14400a830 tx=0x7fe13c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cf58da9e68dc40 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.226+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1440072d0 con 0x7fe13c003970 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.226+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14402f070 con 0x7fe13c003970 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.226+0000 7fe158b7f640 1 -- 172.21.15.72:0/3594417434 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1440076f0 con 0x7fe13c003970 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.226+0000 7fe15ab83640 1 -- 172.21.15.72:0/3594417434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe13c003970 msgr2=0x7fe13c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.226+0000 7fe15ab83640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe13c003970 0x7fe13c003d90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe14400a830 tx=0x7fe13c0057f0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.227+0000 7fe15ab83640 1 -- 172.21.15.72:0/3594417434 shutdown_connections 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.227+0000 7fe15ab83640 1 --2- 172.21.15.72:0/3594417434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe13c003970 0x7fe13c003d90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.227+0000 7fe15ab83640 1 -- 172.21.15.72:0/3594417434 >> 172.21.15.72:0/3594417434 conn(0x7fe1540d3030 msgr2=0x7fe1540d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.227+0000 7fe15ab83640 1 -- 172.21.15.72:0/3594417434 shutdown_connections 2023-12-12T09:55:04.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.227+0000 7fe15ab83640 1 -- 172.21.15.72:0/3594417434 wait complete. 2023-12-12T09:55:04.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.228+0000 7fe15ab83640 1 Processor -- start 2023-12-12T09:55:04.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.229+0000 7fe15ab83640 1 -- start start 2023-12-12T09:55:04.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15ab83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe13c003970 0x7fe1541f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15ab83640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1540b8910 0x7fe154156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15ab83640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1540b5c50 0x7fe1540b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15ab83640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b6610 con 0x7fe1540b5c50 2023-12-12T09:55:04.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15ab83640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b1660 con 0x7fe1540b8910 2023-12-12T09:55:04.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15ab83640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540cc250 con 0x7fe13c003970 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15a382640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1540b5c50 0x7fe1540b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15a382640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1540b5c50 0x7fe1540b60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53378/0 (socket says 172.21.15.72:53378) 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15a382640 1 -- 172.21.15.72:0/2788707865 learned_addr learned my addr 172.21.15.72:0/2788707865 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe159380640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1540b8910 0x7fe154156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe159b81640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe13c003970 0x7fe1541f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15a382640 1 -- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe13c003970 msgr2=0x7fe1541f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15a382640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe13c003970 0x7fe1541f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15a382640 1 -- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1540b8910 msgr2=0x7fe154156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15a382640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1540b8910 0x7fe154156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe15a382640 1 -- 172.21.15.72:0/2788707865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540b48a0 con 0x7fe1540b5c50 2023-12-12T09:55:04.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.230+0000 7fe159380640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1540b8910 0x7fe154156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:04.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.231+0000 7fe15a382640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1540b5c50 0x7fe1540b60d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe15408b4e0 tx=0x7fe14c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:04.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.231+0000 7fe1527fc640 1 -- 172.21.15.72:0/2788707865 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00e8a0 con 0x7fe1540b5c50 2023-12-12T09:55:04.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.231+0000 7fe1527fc640 1 -- 172.21.15.72:0/2788707865 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c00ea00 con 0x7fe1540b5c50 2023-12-12T09:55:04.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.231+0000 7fe1527fc640 1 -- 172.21.15.72:0/2788707865 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00edb0 con 0x7fe1540b5c50 2023-12-12T09:55:04.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.231+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1540b0d70 con 0x7fe1540b5c50 2023-12-12T09:55:04.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.231+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1540b12f0 con 0x7fe1540b5c50 2023-12-12T09:55:04.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.232+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe154074450 con 0x7fe1540b5c50 2023-12-12T09:55:04.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.233+0000 7fe1527fc640 1 -- 172.21.15.72:0/2788707865 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe14c01c400 con 0x7fe1540b5c50 2023-12-12T09:55:04.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.234+0000 7fe1527fc640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe14006ae40 0x7fe14006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.234+0000 7fe1527fc640 1 -- 172.21.15.72:0/2788707865 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe14c089360 con 0x7fe1540b5c50 2023-12-12T09:55:04.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.234+0000 7fe159b81640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe14006ae40 0x7fe14006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.235+0000 7fe159b81640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe14006ae40 0x7fe14006d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe144034000 tx=0x7fe144007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:04.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.239+0000 7fe1527fc640 1 -- 172.21.15.72:0/2788707865 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe14c054530 con 0x7fe1540b5c50 2023-12-12T09:55:04.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.429+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe154072830 con 0x7fe1540b5c50 2023-12-12T09:55:04.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.434+0000 7fe1527fc640 1 -- 172.21.15.72:0/2788707865 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe14c057b50 con 0x7fe1540b5c50 2023-12-12T09:55:04.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe14006ae40 msgr2=0x7fe14006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe14006ae40 0x7fe14006d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe144034000 tx=0x7fe144007000 comp rx=0 tx=0).stop 2023-12-12T09:55:04.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1540b5c50 msgr2=0x7fe1540b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1540b5c50 0x7fe1540b60d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe15408b4e0 tx=0x7fe14c010c20 comp rx=0 tx=0).stop 2023-12-12T09:55:04.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 shutdown_connections 2023-12-12T09:55:04.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe14006ae40 0x7fe14006d300 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1540b5c50 0x7fe1540b60d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1540b8910 0x7fe154156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 --2- 172.21.15.72:0/2788707865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe13c003970 0x7fe1541f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 >> 172.21.15.72:0/2788707865 conn(0x7fe1540d3030 msgr2=0x7fe15406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:04.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 shutdown_connections 2023-12-12T09:55:04.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.437+0000 7fe15ab83640 1 -- 172.21.15.72:0/2788707865 wait complete. 2023-12-12T09:55:04.457 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit' 2023-12-12T09:55:04.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.668+0000 7f2bc34d3640 1 Processor -- start 2023-12-12T09:55:04.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.668+0000 7f2bc34d3640 1 -- start start 2023-12-12T09:55:04.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.669+0000 7f2bc34d3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bbc06c410 0x7f2bbc15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.669+0000 7f2bc34d3640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc06ca40 con 0x7f2bbc15e0e0 2023-12-12T09:55:04.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.669+0000 7f2bc34d3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc15e510 con 0x7f2bbc157a60 2023-12-12T09:55:04.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.669+0000 7f2bc34d3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc071cd0 con 0x7f2bbc06c410 2023-12-12T09:55:04.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.670+0000 7f2bc1a70640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f2bbc15e0e0 0x7f2bbc0cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:37020/0 (socket says 172.21.15.72:37020) 2023-12-12T09:55:04.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.670+0000 7f2bc1a70640 1 -- 172.21.15.72:0/2192064519 learned_addr learned my addr 172.21.15.72:0/2192064519 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.670+0000 7f2bc0a6e640 1 --2- 172.21.15.72:0/2192064519 >> v2:172.21.15.113:3300/0 conn(0x7f2bbc06c410 0x7f2bbc15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.670+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768079909 0 0) 0x7f2bbc06ca40 con 0x7f2bbc15e0e0 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.670+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba4005730 con 0x7f2bbc15e0e0 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 672796258 0 0) 0x7f2ba4005730 con 0x7f2bbc15e0e0 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 >> v2:172.21.15.113:3300/0 conn(0x7f2bbc06c410 msgr2=0x7f2bbc15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 --2- 172.21.15.72:0/2192064519 >> v2:172.21.15.113:3300/0 conn(0x7f2bbc06c410 0x7f2bbc15dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 >> v1:172.21.15.113:6789/0 conn(0x7f2bbc157a60 legacy=0x7f2bbc06de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc070f10 con 0x7f2bbc15e0e0 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f2bb40019f0 con 0x7f2bbc15e0e0 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 >> v1:172.21.15.72:6789/0 conn(0x7f2bbc15e0e0 legacy=0x7f2bbc0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba4005b80 0x7f2ba4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba40069d0 0x7f2ba4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2ba400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc071cd0 con 0x7f2ba4007390 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba400e330 con 0x7f2ba4005b80 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba400e4f0 con 0x7f2ba40069d0 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bc1a70640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2ba400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.671+0000 7f2bc0a6e640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba40069d0 0x7f2ba4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc126f640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba4005b80 0x7f2ba4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc1a70640 1 -- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba40069d0 msgr2=0x7f2ba4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc1a70640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba40069d0 0x7f2ba4006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc1a70640 1 -- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba4005b80 msgr2=0x7f2ba4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc1a70640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba4005b80 0x7f2ba4005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc1a70640 1 -- 172.21.15.72:0/2192064519 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba4005730 con 0x7f2ba4007390 2023-12-12T09:55:04.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc0a6e640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba40069d0 0x7f2ba4006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:04.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc126f640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba4005b80 0x7f2ba4005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc1a70640 1 -- 172.21.15.72:0/2192064519 reap_dead start 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bc1a70640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2ba400db90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400a830 tx=0x7f2ba4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddec47280eb9bf56 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40072d0 con 0x7f2ba4007390 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb402f070 con 0x7f2ba4007390 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.672+0000 7f2bb3fff640 1 -- 172.21.15.72:0/2192064519 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40076b0 con 0x7f2ba4007390 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.673+0000 7f2bc34d3640 1 -- 172.21.15.72:0/2192064519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 msgr2=0x7f2ba400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.673+0000 7f2bc34d3640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2ba400db90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400a830 tx=0x7f2ba4007a00 comp rx=0 tx=0).stop 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.673+0000 7f2bc34d3640 1 -- 172.21.15.72:0/2192064519 shutdown_connections 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.673+0000 7f2bc34d3640 1 --2- 172.21.15.72:0/2192064519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2ba400db90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.673+0000 7f2bc34d3640 1 -- 172.21.15.72:0/2192064519 >> 172.21.15.72:0/2192064519 conn(0x7f2bbc0b0a90 msgr2=0x7f2bbc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.673+0000 7f2bc34d3640 1 -- 172.21.15.72:0/2192064519 shutdown_connections 2023-12-12T09:55:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.673+0000 7f2bc34d3640 1 -- 172.21.15.72:0/2192064519 wait complete. 2023-12-12T09:55:04.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.676+0000 7f2bc34d3640 1 Processor -- start 2023-12-12T09:55:04.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.676+0000 7f2bc34d3640 1 -- start start 2023-12-12T09:55:04.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.677+0000 7f2bc34d3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2bbc15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.677+0000 7f2bc34d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bbc0afdb0 0x7f2bbc0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.677+0000 7f2bc34d3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bbc0b0230 0x7f2bbc0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.677+0000 7f2bc34d3640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0b48e0 con 0x7f2ba4007390 2023-12-12T09:55:04.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.677+0000 7f2bc34d3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0b3c60 con 0x7f2bbc0afdb0 2023-12-12T09:55:04.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.677+0000 7f2bc34d3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0b3f40 con 0x7f2bbc0b0230 2023-12-12T09:55:04.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.678+0000 7f2bc126f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2bbc15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.678+0000 7f2bc126f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2bbc15d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:53400/0 (socket says 172.21.15.72:53400) 2023-12-12T09:55:04.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.678+0000 7f2bc126f640 1 -- 172.21.15.72:0/1230123793 learned_addr learned my addr 172.21.15.72:0/1230123793 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:04.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.678+0000 7f2bc126f640 1 -- 172.21.15.72:0/1230123793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bbc0b0230 msgr2=0x7f2bbc0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.678+0000 7f2bc126f640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bbc0b0230 0x7f2bbc0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.678+0000 7f2bc126f640 1 -- 172.21.15.72:0/1230123793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bbc0afdb0 msgr2=0x7f2bbc0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:04.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.678+0000 7f2bc126f640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bbc0afdb0 0x7f2bbc0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:04.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.678+0000 7f2bc126f640 1 -- 172.21.15.72:0/1230123793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0b4220 con 0x7f2ba4007390 2023-12-12T09:55:04.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.679+0000 7f2bc126f640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2bbc15d410 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00d770 tx=0x7f2bac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:04.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.679+0000 7f2bb1ffb640 1 -- 172.21.15.72:0/1230123793 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac005270 con 0x7f2ba4007390 2023-12-12T09:55:04.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.679+0000 7f2bb1ffb640 1 -- 172.21.15.72:0/1230123793 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac0053d0 con 0x7f2ba4007390 2023-12-12T09:55:04.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.679+0000 7f2bb1ffb640 1 -- 172.21.15.72:0/1230123793 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00b8f0 con 0x7f2ba4007390 2023-12-12T09:55:04.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.679+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc0b4500 con 0x7f2ba4007390 2023-12-12T09:55:04.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.679+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc1f2e20 con 0x7f2ba4007390 2023-12-12T09:55:04.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.681+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc0745d0 con 0x7f2ba4007390 2023-12-12T09:55:04.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.682+0000 7f2bb1ffb640 1 -- 172.21.15.72:0/1230123793 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2bac00ba50 con 0x7f2ba4007390 2023-12-12T09:55:04.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.683+0000 7f2bb1ffb640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2ba806ae90 0x7f2ba806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:04.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.683+0000 7f2bb1ffb640 1 -- 172.21.15.72:0/1230123793 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2bac089390 con 0x7f2ba4007390 2023-12-12T09:55:04.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.683+0000 7f2bc0a6e640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2ba806ae90 0x7f2ba806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:04.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.684+0000 7f2bc0a6e640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2ba806ae90 0x7f2ba806d350 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc070c20 tx=0x7f2bb8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:04.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.687+0000 7f2bb1ffb640 1 -- 172.21.15.72:0/1230123793 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bac0545e0 con 0x7f2ba4007390 2023-12-12T09:55:04.740 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:55:04.748 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:55:04.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:04.950+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit"]} v 0) v1 -- 0x7f2bbc073340 con 0x7f2ba4007390 2023-12-12T09:55:05.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.551+0000 7f2bb1ffb640 1 -- 172.21.15.72:0/1230123793 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit"]}]=0 v179) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2bac019310 con 0x7f2ba4007390 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2ba806ae90 msgr2=0x7f2ba806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2ba806ae90 0x7f2ba806d350 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc070c20 tx=0x7f2bb8008040 comp rx=0 tx=0).stop 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 msgr2=0x7f2bbc15d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2bbc15d410 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00d770 tx=0x7f2bac00dc30 comp rx=0 tx=0).stop 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 shutdown_connections 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2ba806ae90 0x7f2ba806d350 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bbc0b0230 0x7f2bbc0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bbc0afdb0 0x7f2bbc0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 --2- 172.21.15.72:0/1230123793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba4007390 0x7f2bbc15d410 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:05.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 >> 172.21.15.72:0/1230123793 conn(0x7f2bbc0b0a90 msgr2=0x7f2bbc156d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:05.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 shutdown_connections 2023-12-12T09:55:05.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:05.555+0000 7f2bc34d3640 1 -- 172.21.15.72:0/1230123793 wait complete. 2023-12-12T09:55:05.578 INFO:tasks.cephfs_test_runner:That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations). ... FAIL 2023-12-12T09:55:05.578 INFO:tasks.cephfs_test_runner: 2023-12-12T09:55:05.578 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-12T09:55:05.578 INFO:tasks.cephfs_test_runner:FAIL: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations). 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/cephfs/test_strays.py", line 932, in test_dirfrag_limit 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner: self.fail("fragment size exceeded") 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:AssertionError: fragment size exceeded 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner: 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:Ran 2 tests in 112.722s 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner: 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner: 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-12T09:55:05.579 INFO:tasks.cephfs_test_runner:FAIL: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2023-12-12T09:55:05.580 INFO:tasks.cephfs_test_runner:That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations). 2023-12-12T09:55:05.580 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-12T09:55:05.580 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-12T09:55:05.580 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/cephfs/test_strays.py", line 932, in test_dirfrag_limit 2023-12-12T09:55:05.580 INFO:tasks.cephfs_test_runner: self.fail("fragment size exceeded") 2023-12-12T09:55:05.580 INFO:tasks.cephfs_test_runner:AssertionError: fragment size exceeded 2023-12-12T09:55:05.580 INFO:tasks.cephfs_test_runner: 2023-12-12T09:55:05.580 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2023-12-12T09:55:05.680 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=bdeee2297b444d0396a66fc8d3cf51cd Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2023-12-12T09:55:05.681 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-12T09:55:05.691 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-12T09:55:05.714 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-12T09:55:05.715 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:55:05.715 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:55:05.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054258,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-12T09:55:05.801 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-12-12T09:55:05.801 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:55:05.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054258,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-12T09:55:05.856 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:55:05.856 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:55:05.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054290,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-12T09:55:05.915 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-12-12T09:55:05.916 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:55:05.968 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054290,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-12T09:55:05.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:05.968 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:55:06.028 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-12T09:55:06.029 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-12T09:55:06.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.030 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-12T09:55:06.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.127 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-12T09:55:06.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.218 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2023-12-12T09:55:06.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.342 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-12-12T09:55:06.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.390 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2023-12-12T09:55:06.525 INFO:teuthology.orchestra.run.smithi072.stderr:Cannot find device "ceph-brx" 2023-12-12T09:55:06.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:55:06.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.530 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-12-12T09:55:06.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-12T09:55:06.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-12T09:55:06.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.591 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-12T09:55:06.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.679 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-12T09:55:06.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.762 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2023-12-12T09:55:06.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.883 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-12-12T09:55:06.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:55:06.933 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2023-12-12T09:55:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2023-12-12T09:55:07.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:55:07.020 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-12T09:55:07.030 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-12T09:55:07.031 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:55:07.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.251+0000 7fb811a17640 1 Processor -- start 2023-12-12T09:55:07.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.251+0000 7fb811a17640 1 -- start start 2023-12-12T09:55:07.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.253+0000 7fb811a17640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb80c15f260 0x7fb80c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.253+0000 7fb811a17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 0x7fb80c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.253+0000 7fb811a17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb80c070700 0x7fb80c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.253+0000 7fb811a17640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c0cd040 con 0x7fb80c15f260 2023-12-12T09:55:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.253+0000 7fb811a17640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c15f640 con 0x7fb80c070700 2023-12-12T09:55:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.253+0000 7fb811a17640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c070b60 con 0x7fb80c1626b0 2023-12-12T09:55:07.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb80affd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb80c15f260 0x7fb80c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb80affd640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb80c15f260 0x7fb80c0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36870/0 (socket says 172.21.15.72:36870) 2023-12-12T09:55:07.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb80affd640 1 -- 172.21.15.72:0/2043388457 learned_addr learned my addr 172.21.15.72:0/2043388457 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:07.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb8027fc640 1 --2- 172.21.15.72:0/2043388457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 0x7fb80c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb80b7fe640 1 --2- 172.21.15.72:0/2043388457 >> v2:172.21.15.113:3300/0 conn(0x7fb80c070700 0x7fb80c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb80affd640 1 -- 172.21.15.72:0/2043388457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 msgr2=0x7fb80c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:07.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb80affd640 1 --2- 172.21.15.72:0/2043388457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 0x7fb80c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb80affd640 1 -- 172.21.15.72:0/2043388457 >> v2:172.21.15.113:3300/0 conn(0x7fb80c070700 msgr2=0x7fb80c06c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:07.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.254+0000 7fb80affd640 1 --2- 172.21.15.72:0/2043388457 >> v2:172.21.15.113:3300/0 conn(0x7fb80c070700 0x7fb80c06c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.255+0000 7fb80affd640 1 -- 172.21.15.72:0/2043388457 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c0cde40 con 0x7fb80c15f260 2023-12-12T09:55:07.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.255+0000 7fb80b7fe640 1 --2- 172.21.15.72:0/2043388457 >> v2:172.21.15.113:3300/0 conn(0x7fb80c070700 0x7fb80c06c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:07.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.256+0000 7fb80affd640 1 --2- 172.21.15.72:0/2043388457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0ccb00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0b18d0 tx=0x7fb7fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eda371a95492d4b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:07.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.257+0000 7fb80a7fc640 1 -- 172.21.15.72:0/2043388457 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc031640 con 0x7fb80c15f260 2023-12-12T09:55:07.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.257+0000 7fb80a7fc640 1 -- 172.21.15.72:0/2043388457 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0317a0 con 0x7fb80c15f260 2023-12-12T09:55:07.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.257+0000 7fb80a7fc640 1 -- 172.21.15.72:0/2043388457 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc031b30 con 0x7fb80c15f260 2023-12-12T09:55:07.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.257+0000 7fb811a17640 1 -- 172.21.15.72:0/2043388457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 msgr2=0x7fb80c0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.257+0000 7fb811a17640 1 --2- 172.21.15.72:0/2043388457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0ccb00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0b18d0 tx=0x7fb7fc02b8f0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.258+0000 7fb811a17640 1 -- 172.21.15.72:0/2043388457 shutdown_connections 2023-12-12T09:55:07.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.258+0000 7fb811a17640 1 --2- 172.21.15.72:0/2043388457 >> v2:172.21.15.113:3300/0 conn(0x7fb80c070700 0x7fb80c06c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.258+0000 7fb811a17640 1 --2- 172.21.15.72:0/2043388457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 0x7fb80c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.258+0000 7fb811a17640 1 --2- 172.21.15.72:0/2043388457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0ccb00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.258+0000 7fb811a17640 1 -- 172.21.15.72:0/2043388457 >> 172.21.15.72:0/2043388457 conn(0x7fb80c0b0a90 msgr2=0x7fb80c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:07.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.258+0000 7fb811a17640 1 -- 172.21.15.72:0/2043388457 shutdown_connections 2023-12-12T09:55:07.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.258+0000 7fb811a17640 1 -- 172.21.15.72:0/2043388457 wait complete. 2023-12-12T09:55:07.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.260+0000 7fb811a17640 1 Processor -- start 2023-12-12T09:55:07.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.260+0000 7fb811a17640 1 -- start start 2023-12-12T09:55:07.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb811a17640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80c070700 0x7fb80c1f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb811a17640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.263 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb811a17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 0x7fb80c160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb811a17640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b4510 con 0x7fb80c15f260 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb811a17640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b3890 con 0x7fb80c070700 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb811a17640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b3b70 con 0x7fb80c1626b0 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb8027fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb8027fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36872/0 (socket says 172.21.15.72:36872) 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb8027fc640 1 -- 172.21.15.72:0/1809517599 learned_addr learned my addr 172.21.15.72:0/1809517599 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb8027fc640 1 -- 172.21.15.72:0/1809517599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 msgr2=0x7fb80c160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb80affd640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80c070700 0x7fb80c1f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.264 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb8027fc640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 0x7fb80c160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb8027fc640 1 -- 172.21.15.72:0/1809517599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80c070700 msgr2=0x7fb80c1f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb8027fc640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80c070700 0x7fb80c1f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.261+0000 7fb8027fc640 1 -- 172.21.15.72:0/1809517599 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc0094d0 con 0x7fb80c15f260 2023-12-12T09:55:07.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.262+0000 7fb8027fc640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0a9040 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400d5f0 tx=0x7fb7f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:07.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.262+0000 7fb808ff9640 1 -- 172.21.15.72:0/1809517599 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40125f0 con 0x7fb80c15f260 2023-12-12T09:55:07.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.262+0000 7fb808ff9640 1 -- 172.21.15.72:0/1809517599 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4012750 con 0x7fb80c15f260 2023-12-12T09:55:07.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.262+0000 7fb808ff9640 1 -- 172.21.15.72:0/1809517599 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4012b00 con 0x7fb80c15f260 2023-12-12T09:55:07.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.262+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c0b3e50 con 0x7fb80c15f260 2023-12-12T09:55:07.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.262+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c0b34f0 con 0x7fb80c15f260 2023-12-12T09:55:07.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.263+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c0745d0 con 0x7fb80c15f260 2023-12-12T09:55:07.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.264+0000 7fb808ff9640 1 -- 172.21.15.72:0/1809517599 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb7f4012c60 con 0x7fb80c15f260 2023-12-12T09:55:07.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.265+0000 7fb808ff9640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb7ec06ae40 0x7fb7ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.265+0000 7fb808ff9640 1 -- 172.21.15.72:0/1809517599 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb7f40899c0 con 0x7fb80c15f260 2023-12-12T09:55:07.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.265+0000 7fb80affd640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb7ec06ae40 0x7fb7ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.266+0000 7fb80affd640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb7ec06ae40 0x7fb7ec06d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc036000 tx=0x7fb7fc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:07.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.269+0000 7fb808ff9640 1 -- 172.21.15.72:0/1809517599 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7f4054b90 con 0x7fb80c15f260 2023-12-12T09:55:07.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.457+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb80c0b3ce0 con 0x7fb7ec06ae40 2023-12-12T09:55:07.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.464+0000 7fb808ff9640 1 -- 172.21.15.72:0/1809517599 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48424 (secure 0 0 0) 0x7fb80c0b3ce0 con 0x7fb7ec06ae40 2023-12-12T09:55:07.466 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:55:07.467 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:55:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb7ec06ae40 msgr2=0x7fb7ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb7ec06ae40 0x7fb7ec06d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc036000 tx=0x7fb7fc03c040 comp rx=0 tx=0).stop 2023-12-12T09:55:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 msgr2=0x7fb80c0a9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0a9040 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400d5f0 tx=0x7fb7f400dab0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 shutdown_connections 2023-12-12T09:55:07.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb7ec06ae40 0x7fb7ec06d300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb80c1626b0 0x7fb80c160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb80c15f260 0x7fb80c0a9040 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 --2- 172.21.15.72:0/1809517599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb80c070700 0x7fb80c1f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 >> 172.21.15.72:0/1809517599 conn(0x7fb80c0b0a90 msgr2=0x7fb80c162d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:07.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 shutdown_connections 2023-12-12T09:55:07.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.467+0000 7fb811a17640 1 -- 172.21.15.72:0/1809517599 wait complete. 2023-12-12T09:55:07.486 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2023-12-12T09:55:05.601517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256664,"kb_used_data":7212,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718056,"statfs":{"total":773094113280,"available":772831289344,"internally_reserved":0,"allocated":7385088,"data_stored":4117275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-346676,"num_objects":-72,"num_object_clones":0,"num_object_copies":-144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-106,"num_write_kb":-410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001623"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-12-12T09:54:55.873090+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:54:55.873090+0000","last_peered":"2023-12-12T09:54:55.873090+0000","last_clean":"2023-12-12T09:54:55.873090+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:54:55.873090+0000","last_undegraded":"2023-12-12T09:54:55.873090+0000","last_fullsized":"2023-12-12T09:54:55.873090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1310720,"data_stored":871024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0840000000000001,"15min":1.0880000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.075},{"interface":"front","average":{"1min":1.115,"5min":1.131,"15min":1.1339999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.7110000000000001,"5min":1.855,"15min":1.855},"last":1.167}]},{"osd":1,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0189999999999999,"15min":1.018},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.4990000000000001,"5min":1.571,"15min":1.571},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.1439999999999999,"15min":1.145},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5720000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.258}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.367,"15min":1.395},"min":{"1min":0.82099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.78,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.421,"15min":1.4650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6379999999999999,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.2170000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.351,"15min":1.361},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.899,"5min":2.23,"15min":2.23},"last":1.415},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.397,"15min":1.4139999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9850000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.3680000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.411,"15min":1.4359999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.0449999999999999,"5min":2.649,"15min":2.649},"last":1.454},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.4079999999999999,"15min":1.4279999999999999},"min":{"1min":0.71399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7649999999999999,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.325}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.54,"15min":1.5700000000000001},"min":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.784,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5529999999999999,"15min":1.5649999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.141,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.625}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.605,"15min":1.623},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0779999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.599,"15min":1.627},"min":{"1min":0.82599999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9370000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.508}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":281162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.054,"15min":1.046},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1060000000000001,"15min":1.1040000000000001},"min":{"1min":0.76200000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.0069999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.77100000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6579999999999999,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.81399999999999995,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.887,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.1599999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1399999999999999,"15min":1.1339999999999999},"min":{"1min":0.82799999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.5249999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.3280000000000001,"15min":1.3140000000000001},"min":{"1min":1.0389999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.264,"15min":1.2569999999999999},"min":{"1min":0.88600000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.776,"5min":1.994,"15min":1.994},"last":1.323}]},{"osd":4,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.76500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.827,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.496,"5min":1.536,"15min":1.5429999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2890000000000001,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.4870000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3859999999999999,"15min":1.3779999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.98,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.597},{"interface":"front","average":{"1min":1.55,"5min":1.595,"15min":1.603},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":2.1459999999999999,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.748}]},{"osd":7,"last update":"Tue Dec 12 09:54:00 2023","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":1.7030000000000001,"15min":1.7090000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.4119999999999999,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.52,"15min":1.5129999999999999},"min":{"1min":0.98599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.246,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.4590000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":663552,"data_stored":280511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0429999999999999,"15min":1.026},"min":{"1min":0.71799999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0750000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.228,"15min":1.2509999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5249999999999999,"5min":3.694,"15min":3.694},"last":0.93500000000000005}]},{"osd":1,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.3220000000000001,"15min":1.3480000000000001},"min":{"1min":0.76300000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7470000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0880000000000001,"15min":1.0960000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.0900000000000001,"15min":1.0760000000000001},"min":{"1min":0.91400000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.248},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.115,"15min":1.0940000000000001},"min":{"1min":0.874,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.292}]},{"osd":3,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.361,"15min":1.371},"min":{"1min":0.79100000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.74,"5min":3.944,"15min":3.944},"last":1.341},{"interface":"front","average":{"1min":1.327,"5min":1.298,"15min":1.2929999999999999},"min":{"1min":0.92500000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.8260000000000001,"5min":1.925,"15min":1.925},"last":1.468}]},{"osd":4,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.341,"15min":1.335},"min":{"1min":1.0209999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0150000000000001,"5min":2.536,"15min":2.536},"last":1.105},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.53,"15min":1.554},"min":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8420000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.21}]},{"osd":6,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.462,"15min":1.464},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9430000000000001,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.155},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5009999999999999,"15min":1.4990000000000001},"min":{"1min":1.0649999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.379}]},{"osd":7,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.621,"15min":1.633},"min":{"1min":1.095,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9870000000000001,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.409},{"interface":"front","average":{"1min":1.544,"5min":1.647,"15min":1.6639999999999999},"min":{"1min":1.1040000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.048,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.581}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":667648,"data_stored":280076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.095,"15min":1.077},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":5.5430000000000001,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.196,"15min":1.1160000000000001},"min":{"1min":0.66500000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":7.29,"5min":7.29,"15min":7.29},"last":1.147}]},{"osd":1,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.2290000000000001,"15min":1.171},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.3360000000000003,"5min":7.3360000000000003,"15min":7.3360000000000003},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.2949999999999999,"15min":1.288},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.7530000000000001,"5min":6.7530000000000001,"15min":6.7530000000000001},"last":1.5129999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.165,"15min":1.1539999999999999},"min":{"1min":0.67700000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.891,"5min":4.891,"15min":4.891},"last":1.109},{"interface":"front","average":{"1min":1.742,"5min":1.373,"15min":1.3109999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":7.4100000000000001,"5min":7.4100000000000001,"15min":7.4100000000000001},"last":1.423}]},{"osd":3,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3200000000000001,"15min":1.306},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":1.635},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.337,"15min":1.347},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.7409999999999997,"5min":4.7409999999999997,"15min":4.7409999999999997},"last":1.264}]},{"osd":5,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.9379999999999999,"5min":1.502,"15min":1.429},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":1.581},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.508,"15min":1.49},"min":{"1min":0.872,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":6.8499999999999996,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":1.712}]},{"osd":6,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.409,"15min":1.405},"min":{"1min":0.64800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.379},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.619,"15min":1.5620000000000001},"min":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":7.3739999999999997,"5min":7.3739999999999997,"15min":7.3739999999999997},"last":1.6699999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:54:03 2023","interfaces":[{"interface":"back","average":{"1min":1.7030000000000001,"5min":1.5880000000000001,"15min":1.569},"min":{"1min":0.93799999999999994,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.768},{"interface":"front","average":{"1min":1.823,"5min":1.613,"15min":1.5780000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":6.4260000000000002,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.47}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1052672,"data_stored":627471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81200000000000006,"15min":0.80600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.879,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.484,"5min":1.577,"15min":1.577},"last":0.629}]},{"osd":1,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.88500000000000001,"15min":0.875},"min":{"1min":0.54600000000000004,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4630000000000001,"5min":1.548,"15min":1.548},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96999999999999997,"15min":0.97399999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4279999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.98599999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.05,"15min":1.0640000000000001},"min":{"1min":0.42499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.6060000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.3340000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2929999999999999,"15min":1.3},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.645,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.252,"15min":1.262},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6870000000000001,"5min":1.964,"15min":1.964},"last":1.3759999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.2949999999999999,"15min":1.3149999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.669,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.238},{"interface":"front","average":{"1min":1.3,"5min":1.3160000000000001,"15min":1.319},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.885,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.4399999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3280000000000001,"15min":1.3320000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.738,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3169999999999999,"15min":1.327},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6499999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.2749999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3440000000000001,"15min":1.351},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.081},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.387},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.5289999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1028096,"data_stored":627440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.83599999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.89400000000000002,"15min":0.91100000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2230000000000001,"5min":1.752,"15min":1.752},"last":1.508}]},{"osd":1,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.90700000000000003,"15min":0.92300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4550000000000001,"5min":1.603,"15min":1.603},"last":1.026},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.95499999999999996,"15min":0.96999999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4019999999999999,"5min":1.633,"15min":1.633},"last":0.97899999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.89900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.1559999999999999,"15min":1.1839999999999999},"min":{"1min":0.53800000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8879999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.395}]},{"osd":4,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.2989999999999999,"15min":1.3220000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.212},{"interface":"front","average":{"1min":1.216,"5min":1.254,"15min":1.26},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.629,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.3500000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.1870000000000001,"15min":1.2110000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.506,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3220000000000001},"min":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.645,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.304}]},{"osd":6,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.369,"15min":1.3919999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.631,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.2010000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.6140000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.452,"15min":1.48},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6779999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.399,"15min":1.429},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1139999999999999,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.4690000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":279661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.153,"5min":1.486,"15min":1.486},"last":1.085},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90800000000000003,"15min":0.91900000000000004},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3520000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.86899999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.879,"15min":0.875},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.579,"5min":1.712,"15min":1.712},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.99199999999999999,"15min":0.99299999999999999},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.25}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.018,"15min":1.036},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3919999999999999,"5min":1.665,"15min":1.665},"last":1.655},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":0.98999999999999999},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4570000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.212}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.123,"15min":1.1319999999999999},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4570000000000001,"5min":2.056,"15min":2.056},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2929999999999999,"15min":1.3089999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":2.427,"15min":2.427},"last":1.7869999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.363,"15min":1.365},"min":{"1min":1.05,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8129999999999999,"5min":3.476,"15min":3.476},"last":1.288},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.3300000000000001,"15min":1.3500000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.7010000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.3959999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4139999999999999,"15min":1.4279999999999999},"min":{"1min":0.94699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7230000000000001,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.3380000000000001,"15min":1.3620000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.573,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.73}]},{"osd":7,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3839999999999999,"15min":1.3979999999999999},"min":{"1min":0.83199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.694,"5min":3.052,"15min":3.052},"last":1.9950000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4379999999999999,"15min":1.4510000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9079999999999999,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":1.4490000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":869930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3999999999999999,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3420000000000001,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":0.71199999999999997}]},{"osd":2,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.56999999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.165,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0600000000000001,"15min":1.0860000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.5089999999999999,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.98199999999999998},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":1.296,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.661,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.2250000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1779999999999999,"15min":1.1890000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.577,"5min":2.738,"15min":2.738},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.3300000000000001,"15min":1.3560000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.528,"5min":3.875,"15min":3.875},"last":1.397}]},{"osd":5,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2589999999999999,"15min":1.2729999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.625,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2749999999999999,"15min":1.2909999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5680000000000001,"5min":2.54,"15min":2.54},"last":1.514}]},{"osd":6,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.698,"5min":2.117,"15min":2.117},"last":1.129},{"interface":"front","average":{"1min":1.282,"5min":1.4179999999999999,"15min":1.4399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7230000000000001,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":1.141}]},{"osd":7,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3779999999999999,"15min":1.3859999999999999},"min":{"1min":0.84699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7629999999999999,"5min":2.931,"15min":2.931},"last":1.377},{"interface":"front","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6399999999999999,"5min":4.548,"15min":4.548},"last":1.321}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:55:07.488 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:55:07.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.701+0000 7fb9499ec640 1 Processor -- start 2023-12-12T09:55:07.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.701+0000 7fb9499ec640 1 -- start start 2023-12-12T09:55:07.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.703+0000 7fb9499ec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb944072180 0x7fb9440b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.703+0000 7fb9499ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb94406c410 0x7fb94415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.703+0000 7fb9499ec640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9440b3bb0 con 0x7fb944143e00 2023-12-12T09:55:07.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.703+0000 7fb9499ec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9441441e0 con 0x7fb944072180 2023-12-12T09:55:07.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.703+0000 7fb9499ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb944072680 con 0x7fb94406c410 2023-12-12T09:55:07.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.704+0000 7fb9437fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb94406c410 0x7fb94415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.704+0000 7fb93a7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb944072180 0x7fb9440b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.704+0000 7fb9437fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb94406c410 0x7fb94415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:36430/0 (socket says 172.21.15.72:36430) 2023-12-12T09:55:07.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.704+0000 7fb9437fe640 1 -- 172.21.15.72:0/3952780218 learned_addr learned my addr 172.21.15.72:0/3952780218 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:07.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.705+0000 7fb9437fe640 1 -- 172.21.15.72:0/3952780218 >> v2:172.21.15.113:3300/0 conn(0x7fb944072180 msgr2=0x7fb9440b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:07.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.705+0000 7fb9437fe640 1 --2- 172.21.15.72:0/3952780218 >> v2:172.21.15.113:3300/0 conn(0x7fb944072180 0x7fb9440b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.705+0000 7fb9437fe640 1 -- 172.21.15.72:0/3952780218 >> v1:172.21.15.113:6789/0 conn(0x7fb944143e00 legacy=0x7fb9440b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.705+0000 7fb9437fe640 1 -- 172.21.15.72:0/3952780218 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440b1e30 con 0x7fb94406c410 2023-12-12T09:55:07.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.706+0000 7fb9437fe640 1 --2- 172.21.15.72:0/3952780218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb94406c410 0x7fb94415ecb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009e50 tx=0x7fb93c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8d67b472f207a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:07.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.707+0000 7fb9427fc640 1 -- 172.21.15.72:0/3952780218 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c02cb00 con 0x7fb94406c410 2023-12-12T09:55:07.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.707+0000 7fb9427fc640 1 -- 172.21.15.72:0/3952780218 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c02cc60 con 0x7fb94406c410 2023-12-12T09:55:07.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.707+0000 7fb9427fc640 1 -- 172.21.15.72:0/3952780218 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0343f0 con 0x7fb94406c410 2023-12-12T09:55:07.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.708+0000 7fb9499ec640 1 -- 172.21.15.72:0/3952780218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb94406c410 msgr2=0x7fb94415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.708+0000 7fb9499ec640 1 --2- 172.21.15.72:0/3952780218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb94406c410 0x7fb94415ecb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009e50 tx=0x7fb93c02bf60 comp rx=0 tx=0).stop 2023-12-12T09:55:07.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.708+0000 7fb9499ec640 1 -- 172.21.15.72:0/3952780218 shutdown_connections 2023-12-12T09:55:07.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.708+0000 7fb9499ec640 1 --2- 172.21.15.72:0/3952780218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb94406c410 0x7fb94415ecb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.708+0000 7fb9499ec640 1 --2- 172.21.15.72:0/3952780218 >> v2:172.21.15.113:3300/0 conn(0x7fb944072180 0x7fb9440b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.708+0000 7fb9499ec640 1 -- 172.21.15.72:0/3952780218 >> 172.21.15.72:0/3952780218 conn(0x7fb9440b0a90 msgr2=0x7fb9440b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:07.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.708+0000 7fb9499ec640 1 -- 172.21.15.72:0/3952780218 shutdown_connections 2023-12-12T09:55:07.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.708+0000 7fb9499ec640 1 -- 172.21.15.72:0/3952780218 wait complete. 2023-12-12T09:55:07.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.710+0000 7fb9499ec640 1 Processor -- start 2023-12-12T09:55:07.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.710+0000 7fb9499ec640 1 -- start start 2023-12-12T09:55:07.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.711+0000 7fb9499ec640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb94406c410 0x7fb9441300d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.711+0000 7fb9499ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944072180 0x7fb94412e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.711+0000 7fb9499ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb944143e00 0x7fb944155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.711+0000 7fb9499ec640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94412e3e0 con 0x7fb94406c410 2023-12-12T09:55:07.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.711+0000 7fb9499ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944135ce0 con 0x7fb944072180 2023-12-12T09:55:07.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.711+0000 7fb9499ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94412dab0 con 0x7fb944143e00 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb94406c410 0x7fb9441300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb94406c410 0x7fb9441300d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36878/0 (socket says 172.21.15.72:36878) 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 -- 172.21.15.72:0/259372685 learned_addr learned my addr 172.21.15.72:0/259372685 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb93a7fc640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944072180 0x7fb94412e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb9437fe640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb944143e00 0x7fb944155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 -- 172.21.15.72:0/259372685 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb944143e00 msgr2=0x7fb944155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb944143e00 0x7fb944155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 -- 172.21.15.72:0/259372685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944072180 msgr2=0x7fb94412e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944072180 0x7fb94412e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 -- 172.21.15.72:0/259372685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c009b30 con 0x7fb94406c410 2023-12-12T09:55:07.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.712+0000 7fb942ffd640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb94406c410 0x7fb9441300d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c400 tx=0x7fb93400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:07.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.713+0000 7fb940ff9640 1 -- 172.21.15.72:0/259372685 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9340052d0 con 0x7fb94406c410 2023-12-12T09:55:07.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.713+0000 7fb940ff9640 1 -- 172.21.15.72:0/259372685 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934005450 con 0x7fb94406c410 2023-12-12T09:55:07.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.713+0000 7fb940ff9640 1 -- 172.21.15.72:0/259372685 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934007260 con 0x7fb94406c410 2023-12-12T09:55:07.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.713+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94412dd90 con 0x7fb94406c410 2023-12-12T09:55:07.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.713+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb944135900 con 0x7fb94406c410 2023-12-12T09:55:07.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.714+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9440745d0 con 0x7fb94406c410 2023-12-12T09:55:07.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.715+0000 7fb940ff9640 1 -- 172.21.15.72:0/259372685 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9340073c0 con 0x7fb94406c410 2023-12-12T09:55:07.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.716+0000 7fb940ff9640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb92406ae70 0x7fb92406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:07.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.716+0000 7fb940ff9640 1 -- 172.21.15.72:0/259372685 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb934088840 con 0x7fb94406c410 2023-12-12T09:55:07.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.716+0000 7fb93a7fc640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb92406ae70 0x7fb92406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:07.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.717+0000 7fb93a7fc640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb92406ae70 0x7fb92406d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb94415b350 tx=0x7fb92c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:07.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.720+0000 7fb940ff9640 1 -- 172.21.15.72:0/259372685 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb934053ab0 con 0x7fb94406c410 2023-12-12T09:55:07.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.905+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9440b23f0 con 0x7fb92406ae70 2023-12-12T09:55:07.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.910+0000 7fb940ff9640 1 -- 172.21.15.72:0/259372685 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48383 (secure 0 0 0) 0x7fb92c011970 con 0x7fb92406ae70 2023-12-12T09:55:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:55:07.914 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:55:07.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb92406ae70 msgr2=0x7fb92406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb92406ae70 0x7fb92406d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb94415b350 tx=0x7fb92c008040 comp rx=0 tx=0).stop 2023-12-12T09:55:07.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb94406c410 msgr2=0x7fb9441300d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:07.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb94406c410 0x7fb9441300d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c400 tx=0x7fb93400c8c0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 shutdown_connections 2023-12-12T09:55:07.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb92406ae70 0x7fb92406d330 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb944143e00 0x7fb944155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944072180 0x7fb94412e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 --2- 172.21.15.72:0/259372685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb94406c410 0x7fb9441300d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:07.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 >> 172.21.15.72:0/259372685 conn(0x7fb9440b0a90 msgr2=0x7fb9440af890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:07.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 shutdown_connections 2023-12-12T09:55:07.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:07.913+0000 7fb9499ec640 1 -- 172.21.15.72:0/259372685 wait complete. 2023-12-12T09:55:07.933 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2023-12-12T09:55:07.602082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":256028,"kb_used_data":6576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718692,"statfs":{"total":773094113280,"available":772831940608,"internally_reserved":0,"allocated":6733824,"data_stored":3554051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-346676,"num_objects":-72,"num_object_clones":0,"num_object_copies":-144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-106,"num_write_kb":-410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.232473"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":197,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-12-12T09:55:02.535538+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:55:02.535538+0000","last_peered":"2023-12-12T09:55:02.535538+0000","last_clean":"2023-12-12T09:55:02.535538+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:55:02.535538+0000","last_undegraded":"2023-12-12T09:55:02.535538+0000","last_fullsized":"2023-12-12T09:55:02.535538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":883740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0840000000000001,"15min":1.0880000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.288},{"interface":"front","average":{"1min":1.115,"5min":1.131,"15min":1.1339999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.7110000000000001,"5min":1.855,"15min":1.855},"last":1.6950000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0189999999999999,"15min":1.018},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.4990000000000001,"5min":1.571,"15min":1.571},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.1439999999999999,"15min":1.145},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5720000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.365}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.367,"15min":1.395},"min":{"1min":0.82099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.78,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.8420000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.421,"15min":1.4650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6379999999999999,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.0609999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.351,"15min":1.361},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.899,"5min":2.23,"15min":2.23},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.397,"15min":1.4139999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9850000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.794}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.411,"15min":1.4359999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.0449999999999999,"5min":2.649,"15min":2.649},"last":1.417},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.4079999999999999,"15min":1.4279999999999999},"min":{"1min":0.71399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7649999999999999,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.95599999999999996}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.54,"15min":1.5700000000000001},"min":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.784,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5529999999999999,"15min":1.5649999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.141,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.516}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.605,"15min":1.623},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0779999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.921},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.599,"15min":1.627},"min":{"1min":0.82599999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9370000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.6259999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0529999999999999,"15min":1.0449999999999999},"min":{"1min":0.76000000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5820000000000001,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.115,"15min":1.107},"min":{"1min":0.754,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.702,"5min":1.865,"15min":1.865},"last":0.97899999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2330000000000001,"15min":1.2270000000000001},"min":{"1min":0.81999999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7250000000000001,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.95699999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.5940000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.4630000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.139,"15min":1.133},"min":{"1min":0.747,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.393,"5min":1.859,"15min":1.859},"last":1.107},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2769999999999999,"15min":1.2889999999999999},"min":{"1min":0.754,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.5589999999999999,"5min":2.028,"15min":2.028},"last":1.0429999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3200000000000001,"15min":1.3109999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.26,"15min":1.256},"min":{"1min":0.90900000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7210000000000001,"5min":1.994,"15min":1.994},"last":1.4259999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.379,"15min":1.393},"min":{"1min":0.92000000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6799999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.163},{"interface":"front","average":{"1min":1.458,"5min":1.5189999999999999,"15min":1.5369999999999999},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.1269999999999998,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.605}]},{"osd":5,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3819999999999999,"15min":1.377},"min":{"1min":1.0229999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.6679999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.996,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":1.7829999999999999,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.5449999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.6619999999999999,"15min":1.696},"min":{"1min":0.93999999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2269999999999999,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.679},{"interface":"front","average":{"1min":1.504,"5min":1.5189999999999999,"15min":1.5129999999999999},"min":{"1min":1.288,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8200000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.643}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0429999999999999,"15min":1.026},"min":{"1min":0.71799999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0750000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.228,"15min":1.2509999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5249999999999999,"5min":3.694,"15min":3.694},"last":1.2270000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.3220000000000001,"15min":1.3480000000000001},"min":{"1min":0.76300000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7470000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0880000000000001,"15min":1.0960000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.88200000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.0900000000000001,"15min":1.0760000000000001},"min":{"1min":0.91400000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.115,"15min":1.0940000000000001},"min":{"1min":0.874,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.7470000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.361,"15min":1.371},"min":{"1min":0.79100000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.74,"5min":3.944,"15min":3.944},"last":1.498},{"interface":"front","average":{"1min":1.327,"5min":1.298,"15min":1.2929999999999999},"min":{"1min":0.92500000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.8260000000000001,"5min":1.925,"15min":1.925},"last":1.462}]},{"osd":4,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.341,"15min":1.335},"min":{"1min":1.0209999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0150000000000001,"5min":2.536,"15min":2.536},"last":1.623},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.53,"15min":1.554},"min":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8420000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.671}]},{"osd":6,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.462,"15min":1.464},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9430000000000001,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.571},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5009999999999999,"15min":1.4990000000000001},"min":{"1min":1.0649999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.694}]},{"osd":7,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.621,"15min":1.633},"min":{"1min":1.095,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9870000000000001,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.647,"15min":1.6639999999999999},"min":{"1min":1.1040000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.048,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.9450000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":298236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.101,"15min":1.079},"min":{"1min":0.626,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.639,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.224,"15min":1.1259999999999999},"min":{"1min":0.745,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7410000000000001,"5min":7.29,"15min":7.29},"last":0.998}]},{"osd":1,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2529999999999999,"15min":1.179},"min":{"1min":0.91700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.587,"5min":7.3360000000000003,"15min":7.3360000000000003},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.282,"15min":1.284},"min":{"1min":0.88900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":6.7530000000000001,"15min":6.7530000000000001},"last":1.446}]},{"osd":2,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.165,"15min":1.1539999999999999},"min":{"1min":0.69899999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6339999999999999,"5min":4.891,"15min":4.891},"last":1.472},{"interface":"front","average":{"1min":1.294,"5min":1.375,"15min":1.3120000000000001},"min":{"1min":0.747,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8169999999999999,"5min":7.4100000000000001,"15min":7.4100000000000001},"last":1.292}]},{"osd":3,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3200000000000001,"15min":1.306},"min":{"1min":0.81100000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.635,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":1.389},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.323,"15min":1.3420000000000001},"min":{"1min":0.74099999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0720000000000001,"5min":4.7409999999999997,"15min":4.7409999999999997},"last":1.3600000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.4970000000000001,"15min":1.427},"min":{"1min":0.81399999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8899999999999999,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":1.569},{"interface":"front","average":{"1min":1.409,"5min":1.4930000000000001,"15min":1.4850000000000001},"min":{"1min":0.89600000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8959999999999999,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.405,"15min":1.4039999999999999},"min":{"1min":0.86799999999999999,"5min":0.629,"15min":0.629},"max":{"1min":2.1880000000000002,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.141},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.6080000000000001,"15min":1.5580000000000001},"min":{"1min":0.95299999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9730000000000001,"5min":7.3739999999999997,"15min":7.3739999999999997},"last":1.5089999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.587,"15min":1.569},"min":{"1min":1.046,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.0099999999999998,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.6100000000000001,"15min":1.577},"min":{"1min":1.137,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.23,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.542}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82099999999999995,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4510000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93000000000000005,"15min":0.93999999999999995},"min":{"1min":0.629,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.206,"5min":1.577,"15min":1.577},"last":0.71799999999999997}]},{"osd":1,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.873,"15min":0.871},"min":{"1min":0.55600000000000005,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1499999999999999,"5min":1.548,"15min":1.548},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.105,"5min":0.996,"15min":0.98199999999999998},"min":{"1min":0.66700000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.89100000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.048,"15min":1.0629999999999999},"min":{"1min":0.67500000000000004,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4710000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.504,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.93899999999999995}]},{"osd":4,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.81599999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2360000000000002,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.135},{"interface":"front","average":{"1min":1.367,"5min":1.272,"15min":1.268},"min":{"1min":1.046,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.782,"5min":1.964,"15min":1.964},"last":1.3109999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3009999999999999,"15min":1.3169999999999999},"min":{"1min":0.97899999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7290000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.429},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.327,"15min":1.323},"min":{"1min":0.83699999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.748,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.6080000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.321,"15min":1.329},"min":{"1min":0.68200000000000005,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.919,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.474},{"interface":"front","average":{"1min":1.385,"5min":1.327,"15min":1.3300000000000001},"min":{"1min":0.99399999999999999,"5min":0.627,"15min":0.627},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.216}]},{"osd":7,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3580000000000001,"15min":1.3560000000000001},"min":{"1min":1.081,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8080000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.103},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.022,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.831,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.2689999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.83599999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.89400000000000002,"15min":0.91100000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2230000000000001,"5min":1.752,"15min":1.752},"last":0.29699999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.90700000000000003,"15min":0.92300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4550000000000001,"5min":1.603,"15min":1.603},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.95499999999999996,"15min":0.96999999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4019999999999999,"5min":1.633,"15min":1.633},"last":0.39400000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.89900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.753},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.1559999999999999,"15min":1.1839999999999999},"min":{"1min":0.53800000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8879999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.65900000000000003}]},{"osd":4,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.2989999999999999,"15min":1.3220000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.216,"5min":1.254,"15min":1.26},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.629,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.1870000000000001,"15min":1.2110000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.506,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3220000000000001},"min":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.645,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.72999999999999998}]},{"osd":6,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.369,"15min":1.3919999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.631,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.2010000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.153}]},{"osd":7,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.452,"15min":1.48},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6779999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.399,"15min":1.429},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1139999999999999,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.1839999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":712704,"data_stored":292341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.153,"5min":1.486,"15min":1.486},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90800000000000003,"15min":0.91900000000000004},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3520000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.95299999999999996}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.879,"15min":0.875},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.579,"5min":1.712,"15min":1.712},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.99199999999999999,"15min":0.99299999999999999},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.89200000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.018,"15min":1.036},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3919999999999999,"5min":1.665,"15min":1.665},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":0.98999999999999999},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4570000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.123,"15min":1.1319999999999999},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4570000000000001,"5min":2.056,"15min":2.056},"last":1.165},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2929999999999999,"15min":1.3089999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":2.427,"15min":2.427},"last":0.998}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.363,"15min":1.365},"min":{"1min":1.05,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8129999999999999,"5min":3.476,"15min":3.476},"last":1.218},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.3300000000000001,"15min":1.3500000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.7010000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.3759999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4139999999999999,"15min":1.4279999999999999},"min":{"1min":0.94699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7230000000000001,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.556},{"interface":"front","average":{"1min":1.198,"5min":1.3380000000000001,"15min":1.3620000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.573,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.5149999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3839999999999999,"15min":1.3979999999999999},"min":{"1min":0.83199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.694,"5min":3.052,"15min":3.052},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4379999999999999,"15min":1.4510000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9079999999999999,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":1.327}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1314816,"data_stored":882714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3999999999999999,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3420000000000001,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":0.83999999999999997}]},{"osd":2,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.56999999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.165,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":1.137},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0600000000000001,"15min":1.0860000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.5089999999999999,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":1.3200000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.98199999999999998},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":1.296,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.661,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.397}]},{"osd":4,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1779999999999999,"15min":1.1890000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.577,"5min":2.738,"15min":2.738},"last":1.53},{"interface":"front","average":{"1min":1.171,"5min":1.3300000000000001,"15min":1.3560000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.528,"5min":3.875,"15min":3.875},"last":1.454}]},{"osd":5,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2589999999999999,"15min":1.2729999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.625,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.214},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2749999999999999,"15min":1.2909999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5680000000000001,"5min":2.54,"15min":2.54},"last":1.268}]},{"osd":6,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.698,"5min":2.117,"15min":2.117},"last":1.732},{"interface":"front","average":{"1min":1.282,"5min":1.4179999999999999,"15min":1.4399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7230000000000001,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":1.5860000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3779999999999999,"15min":1.3859999999999999},"min":{"1min":0.84699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7629999999999999,"5min":2.931,"15min":2.931},"last":1.806},{"interface":"front","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6399999999999999,"5min":4.548,"15min":4.548},"last":1.639}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:55:07.935 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-12T09:55:07.935 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:55:08.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.139+0000 7fd3870b4640 1 Processor -- start 2023-12-12T09:55:08.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.139+0000 7fd3870b4640 1 -- start start 2023-12-12T09:55:08.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.140+0000 7fd3870b4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd38006c410 0x7fd38015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.141+0000 7fd3870b4640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38006ca40 con 0x7fd380157a60 2023-12-12T09:55:08.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.141+0000 7fd3870b4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38015e510 con 0x7fd38015e0e0 2023-12-12T09:55:08.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.141+0000 7fd3870b4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd380071cd0 con 0x7fd38006c410 2023-12-12T09:55:08.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.141+0000 7fd384e50640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd380157a60 0x7fd38006de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43102/0 (socket says 172.21.15.72:43102) 2023-12-12T09:55:08.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.141+0000 7fd384e50640 1 -- 172.21.15.72:0/2671262368 learned_addr learned my addr 172.21.15.72:0/2671262368 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:08.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.141+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584255757 0 0) 0x7fd38006ca40 con 0x7fd380157a60 2023-12-12T09:55:08.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.141+0000 7fd37ffff640 1 --2- 172.21.15.72:0/2671262368 >> v2:172.21.15.113:3300/0 conn(0x7fd38006c410 0x7fd38015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.141+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd370003520 con 0x7fd380157a60 2023-12-12T09:55:08.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3855032842 0 0) 0x7fd370003520 con 0x7fd380157a60 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 >> v2:172.21.15.113:3300/0 conn(0x7fd38006c410 msgr2=0x7fd38015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 --2- 172.21.15.72:0/2671262368 >> v2:172.21.15.113:3300/0 conn(0x7fd38006c410 0x7fd38015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 >> v1:172.21.15.113:6789/0 conn(0x7fd38015e0e0 legacy=0x7fd3800cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd380070f10 con 0x7fd380157a60 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fd374001670 con 0x7fd380157a60 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 >> v1:172.21.15.72:6789/0 conn(0x7fd380157a60 legacy=0x7fd38006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd370003970 0x7fd370003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3700047c0 0x7fd370004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd37000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380071cd0 con 0x7fd3700052a0 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37000c150 con 0x7fd3700047c0 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37000c310 con 0x7fd370003970 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd385651640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd37000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd384e50640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd370003970 0x7fd370003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.142+0000 7fd37ffff640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3700047c0 0x7fd370004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.143+0000 7fd385651640 1 -- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd370003970 msgr2=0x7fd370003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.143+0000 7fd385651640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd370003970 0x7fd370003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.143+0000 7fd385651640 1 -- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3700047c0 msgr2=0x7fd370004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:08.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.143+0000 7fd385651640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3700047c0 0x7fd370004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.143+0000 7fd385651640 1 -- 172.21.15.72:0/2671262368 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370003520 con 0x7fd3700052a0 2023-12-12T09:55:08.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.143+0000 7fd384e50640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd370003970 0x7fd370003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:08.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.143+0000 7fd385651640 1 -- 172.21.15.72:0/2671262368 reap_dead start 2023-12-12T09:55:08.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.144+0000 7fd385651640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd37000b980 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd378004040 tx=0x7fd3700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4edb93532a600a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:08.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.144+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37802f710 con 0x7fd3700052a0 2023-12-12T09:55:08.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.144+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37802f870 con 0x7fd3700052a0 2023-12-12T09:55:08.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.144+0000 7fd37f7fe640 1 -- 172.21.15.72:0/2671262368 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37802fc00 con 0x7fd3700052a0 2023-12-12T09:55:08.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.144+0000 7fd3870b4640 1 -- 172.21.15.72:0/2671262368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 msgr2=0x7fd37000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.144+0000 7fd3870b4640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd37000b980 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd378004040 tx=0x7fd3700057f0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.145+0000 7fd3870b4640 1 -- 172.21.15.72:0/2671262368 shutdown_connections 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.145+0000 7fd3870b4640 1 --2- 172.21.15.72:0/2671262368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd37000b980 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.145+0000 7fd3870b4640 1 -- 172.21.15.72:0/2671262368 >> 172.21.15.72:0/2671262368 conn(0x7fd3800b0a90 msgr2=0x7fd3800b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.145+0000 7fd3870b4640 1 -- 172.21.15.72:0/2671262368 shutdown_connections 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.145+0000 7fd3870b4640 1 -- 172.21.15.72:0/2671262368 wait complete. 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.146+0000 7fd3870b4640 1 Processor -- start 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.147+0000 7fd3870b4640 1 -- start start 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.147+0000 7fd3870b4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd38013e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd3870b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd380156190 0x7fd3801565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd3870b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3800b2830 0x7fd3800b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd3870b4640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800b31f0 con 0x7fd3700052a0 2023-12-12T09:55:08.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd3870b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3801550f0 con 0x7fd3800b2830 2023-12-12T09:55:08.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd3870b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3801553d0 con 0x7fd380156190 2023-12-12T09:55:08.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd38013e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd38013e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36892/0 (socket says 172.21.15.72:36892) 2023-12-12T09:55:08.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 -- 172.21.15.72:0/4093163077 learned_addr learned my addr 172.21.15.72:0/4093163077 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:08.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 -- 172.21.15.72:0/4093163077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd380156190 msgr2=0x7fd3801565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd380156190 0x7fd3801565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 -- 172.21.15.72:0/4093163077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3800b2830 msgr2=0x7fd3800b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3800b2830 0x7fd3800b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 -- 172.21.15.72:0/4093163077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3801556b0 con 0x7fd3700052a0 2023-12-12T09:55:08.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.148+0000 7fd384e50640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd38013e080 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd37400d810 tx=0x7fd37400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:08.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.149+0000 7fd37d7fa640 1 -- 172.21.15.72:0/4093163077 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3740074e0 con 0x7fd3700052a0 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.149+0000 7fd37d7fa640 1 -- 172.21.15.72:0/4093163077 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd374007640 con 0x7fd3700052a0 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.149+0000 7fd37d7fa640 1 -- 172.21.15.72:0/4093163077 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374004a20 con 0x7fd3700052a0 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.149+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd380155990 con 0x7fd3700052a0 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.149+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38013e5c0 con 0x7fd3700052a0 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.150+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3800745d0 con 0x7fd3700052a0 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.151+0000 7fd37d7fa640 1 -- 172.21.15.72:0/4093163077 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd374004b80 con 0x7fd3700052a0 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.152+0000 7fd37d7fa640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd36406ae40 0x7fd36406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.152+0000 7fd37d7fa640 1 -- 172.21.15.72:0/4093163077 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd374088e80 con 0x7fd3700052a0 2023-12-12T09:55:08.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.152+0000 7fd37ffff640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd36406ae40 0x7fd36406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.153+0000 7fd37ffff640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd36406ae40 0x7fd36406d300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd36c006a90 tx=0x7fd36c006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:08.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.156+0000 7fd37d7fa640 1 -- 172.21.15.72:0/4093163077 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd374054050 con 0x7fd3700052a0 2023-12-12T09:55:08.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.341+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd380155540 con 0x7fd36406ae40 2023-12-12T09:55:08.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.345+0000 7fd37d7fa640 1 -- 172.21.15.72:0/4093163077 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48383 (secure 0 0 0) 0x7fd380155540 con 0x7fd36406ae40 2023-12-12T09:55:08.347 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:55:08.348 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:55:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.347+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd36406ae40 msgr2=0x7fd36406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.347+0000 7fd3870b4640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd36406ae40 0x7fd36406d300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd36c006a90 tx=0x7fd36c006a20 comp rx=0 tx=0).stop 2023-12-12T09:55:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 msgr2=0x7fd38013e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd38013e080 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd37400d810 tx=0x7fd37400dcd0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 shutdown_connections 2023-12-12T09:55:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd36406ae40 0x7fd36406d300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3800b2830 0x7fd3800b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd380156190 0x7fd3801565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 --2- 172.21.15.72:0/4093163077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3700052a0 0x7fd38013e080 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 >> 172.21.15.72:0/4093163077 conn(0x7fd3800b0a90 msgr2=0x7fd38006d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 shutdown_connections 2023-12-12T09:55:08.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.348+0000 7fd3870b4640 1 -- 172.21.15.72:0/4093163077 wait complete. 2023-12-12T09:55:08.369 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2023-12-12T09:55:07.602082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":256028,"kb_used_data":6576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718692,"statfs":{"total":773094113280,"available":772831940608,"internally_reserved":0,"allocated":6733824,"data_stored":3554051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-346676,"num_objects":-72,"num_object_clones":0,"num_object_copies":-144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-106,"num_write_kb":-410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.232473"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":197,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-12-12T09:55:02.535538+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:55:02.535538+0000","last_peered":"2023-12-12T09:55:02.535538+0000","last_clean":"2023-12-12T09:55:02.535538+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:55:02.535538+0000","last_undegraded":"2023-12-12T09:55:02.535538+0000","last_fullsized":"2023-12-12T09:55:02.535538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":883740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0840000000000001,"15min":1.0880000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.288},{"interface":"front","average":{"1min":1.115,"5min":1.131,"15min":1.1339999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.7110000000000001,"5min":1.855,"15min":1.855},"last":1.6950000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0189999999999999,"15min":1.018},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.4990000000000001,"5min":1.571,"15min":1.571},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.1439999999999999,"15min":1.145},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5720000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.365}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.367,"15min":1.395},"min":{"1min":0.82099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.78,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.8420000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.421,"15min":1.4650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6379999999999999,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.0609999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.351,"15min":1.361},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.899,"5min":2.23,"15min":2.23},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.397,"15min":1.4139999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9850000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.794}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.411,"15min":1.4359999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.0449999999999999,"5min":2.649,"15min":2.649},"last":1.417},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.4079999999999999,"15min":1.4279999999999999},"min":{"1min":0.71399999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7649999999999999,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.95599999999999996}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.54,"15min":1.5700000000000001},"min":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.784,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5529999999999999,"15min":1.5649999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.141,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.516}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.605,"15min":1.623},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0779999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.921},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.599,"15min":1.627},"min":{"1min":0.82599999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9370000000000001,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.6259999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0529999999999999,"15min":1.0449999999999999},"min":{"1min":0.76000000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5820000000000001,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.115,"15min":1.107},"min":{"1min":0.754,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.702,"5min":1.865,"15min":1.865},"last":0.97899999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2330000000000001,"15min":1.2270000000000001},"min":{"1min":0.81999999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7250000000000001,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.95699999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.5940000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.4630000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.139,"15min":1.133},"min":{"1min":0.747,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.393,"5min":1.859,"15min":1.859},"last":1.107},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2769999999999999,"15min":1.2889999999999999},"min":{"1min":0.754,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.5589999999999999,"5min":2.028,"15min":2.028},"last":1.0429999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3200000000000001,"15min":1.3109999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.26,"15min":1.256},"min":{"1min":0.90900000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7210000000000001,"5min":1.994,"15min":1.994},"last":1.4259999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.379,"15min":1.393},"min":{"1min":0.92000000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6799999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.163},{"interface":"front","average":{"1min":1.458,"5min":1.5189999999999999,"15min":1.5369999999999999},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.1269999999999998,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.605}]},{"osd":5,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3819999999999999,"15min":1.377},"min":{"1min":1.0229999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.6679999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.996,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":1.7829999999999999,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.5449999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.6619999999999999,"15min":1.696},"min":{"1min":0.93999999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2269999999999999,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.679},{"interface":"front","average":{"1min":1.504,"5min":1.5189999999999999,"15min":1.5129999999999999},"min":{"1min":1.288,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8200000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.643}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0429999999999999,"15min":1.026},"min":{"1min":0.71799999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0750000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.228,"15min":1.2509999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5249999999999999,"5min":3.694,"15min":3.694},"last":1.2270000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.3220000000000001,"15min":1.3480000000000001},"min":{"1min":0.76300000000000001,"5min":0.748,"15min":0.748},"max":{"1min":1.7470000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0880000000000001,"15min":1.0960000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.88200000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.0900000000000001,"15min":1.0760000000000001},"min":{"1min":0.91400000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.115,"15min":1.0940000000000001},"min":{"1min":0.874,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.7470000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.361,"15min":1.371},"min":{"1min":0.79100000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.74,"5min":3.944,"15min":3.944},"last":1.498},{"interface":"front","average":{"1min":1.327,"5min":1.298,"15min":1.2929999999999999},"min":{"1min":0.92500000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.8260000000000001,"5min":1.925,"15min":1.925},"last":1.462}]},{"osd":4,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.341,"15min":1.335},"min":{"1min":1.0209999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0150000000000001,"5min":2.536,"15min":2.536},"last":1.623},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.53,"15min":1.554},"min":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8420000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.671}]},{"osd":6,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.462,"15min":1.464},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9430000000000001,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.571},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5009999999999999,"15min":1.4990000000000001},"min":{"1min":1.0649999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.694}]},{"osd":7,"last update":"Tue Dec 12 09:54:05 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.621,"15min":1.633},"min":{"1min":1.095,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.9870000000000001,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.647,"15min":1.6639999999999999},"min":{"1min":1.1040000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":2.048,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.9450000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":298236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.101,"15min":1.079},"min":{"1min":0.626,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.639,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.224,"15min":1.1259999999999999},"min":{"1min":0.745,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7410000000000001,"5min":7.29,"15min":7.29},"last":0.998}]},{"osd":1,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2529999999999999,"15min":1.179},"min":{"1min":0.91700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.587,"5min":7.3360000000000003,"15min":7.3360000000000003},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.282,"15min":1.284},"min":{"1min":0.88900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":6.7530000000000001,"15min":6.7530000000000001},"last":1.446}]},{"osd":2,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.165,"15min":1.1539999999999999},"min":{"1min":0.69899999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6339999999999999,"5min":4.891,"15min":4.891},"last":1.472},{"interface":"front","average":{"1min":1.294,"5min":1.375,"15min":1.3120000000000001},"min":{"1min":0.747,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8169999999999999,"5min":7.4100000000000001,"15min":7.4100000000000001},"last":1.292}]},{"osd":3,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3200000000000001,"15min":1.306},"min":{"1min":0.81100000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.635,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":1.389},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.323,"15min":1.3420000000000001},"min":{"1min":0.74099999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0720000000000001,"5min":4.7409999999999997,"15min":4.7409999999999997},"last":1.3600000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.4970000000000001,"15min":1.427},"min":{"1min":0.81399999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8899999999999999,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":1.569},{"interface":"front","average":{"1min":1.409,"5min":1.4930000000000001,"15min":1.4850000000000001},"min":{"1min":0.89600000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8959999999999999,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.405,"15min":1.4039999999999999},"min":{"1min":0.86799999999999999,"5min":0.629,"15min":0.629},"max":{"1min":2.1880000000000002,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.141},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.6080000000000001,"15min":1.5580000000000001},"min":{"1min":0.95299999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9730000000000001,"5min":7.3739999999999997,"15min":7.3739999999999997},"last":1.5089999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.587,"15min":1.569},"min":{"1min":1.046,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.0099999999999998,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.6100000000000001,"15min":1.577},"min":{"1min":1.137,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.23,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.542}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82099999999999995,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4510000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93000000000000005,"15min":0.93999999999999995},"min":{"1min":0.629,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.206,"5min":1.577,"15min":1.577},"last":0.71799999999999997}]},{"osd":1,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.873,"15min":0.871},"min":{"1min":0.55600000000000005,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1499999999999999,"5min":1.548,"15min":1.548},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.105,"5min":0.996,"15min":0.98199999999999998},"min":{"1min":0.66700000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.89100000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.048,"15min":1.0629999999999999},"min":{"1min":0.67500000000000004,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4710000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.504,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.93899999999999995}]},{"osd":4,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.81599999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2360000000000002,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.135},{"interface":"front","average":{"1min":1.367,"5min":1.272,"15min":1.268},"min":{"1min":1.046,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.782,"5min":1.964,"15min":1.964},"last":1.3109999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3009999999999999,"15min":1.3169999999999999},"min":{"1min":0.97899999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7290000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.429},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.327,"15min":1.323},"min":{"1min":0.83699999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.748,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.6080000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.321,"15min":1.329},"min":{"1min":0.68200000000000005,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.919,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.474},{"interface":"front","average":{"1min":1.385,"5min":1.327,"15min":1.3300000000000001},"min":{"1min":0.99399999999999999,"5min":0.627,"15min":0.627},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.216}]},{"osd":7,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3580000000000001,"15min":1.3560000000000001},"min":{"1min":1.081,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8080000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.103},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.022,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.831,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.2689999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.83599999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3899999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.89400000000000002,"15min":0.91100000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2230000000000001,"5min":1.752,"15min":1.752},"last":0.29699999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.90700000000000003,"15min":0.92300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4550000000000001,"5min":1.603,"15min":1.603},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.95499999999999996,"15min":0.96999999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4019999999999999,"5min":1.633,"15min":1.633},"last":0.39400000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.89900000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.753},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.1559999999999999,"15min":1.1839999999999999},"min":{"1min":0.53800000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8879999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.65900000000000003}]},{"osd":4,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.2989999999999999,"15min":1.3220000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.216,"5min":1.254,"15min":1.26},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.629,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.1870000000000001,"15min":1.2110000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.506,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3220000000000001},"min":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.645,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.72999999999999998}]},{"osd":6,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.369,"15min":1.3919999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.631,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3640000000000001,"15min":1.3939999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.2010000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.153}]},{"osd":7,"last update":"Tue Dec 12 09:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.452,"15min":1.48},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6779999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.399,"15min":1.429},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1139999999999999,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.1839999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":712704,"data_stored":292341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.153,"5min":1.486,"15min":1.486},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90800000000000003,"15min":0.91900000000000004},"min":{"1min":0.502,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3520000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.95299999999999996}]},{"osd":2,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.879,"15min":0.875},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.579,"5min":1.712,"15min":1.712},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.99199999999999999,"15min":0.99299999999999999},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.89200000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.018,"15min":1.036},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3919999999999999,"5min":1.665,"15min":1.665},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":0.98999999999999999},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4570000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.123,"15min":1.1319999999999999},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4570000000000001,"5min":2.056,"15min":2.056},"last":1.165},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2929999999999999,"15min":1.3089999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.607,"5min":2.427,"15min":2.427},"last":0.998}]},{"osd":5,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.363,"15min":1.365},"min":{"1min":1.05,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8129999999999999,"5min":3.476,"15min":3.476},"last":1.218},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.3300000000000001,"15min":1.3500000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.7010000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.3759999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4139999999999999,"15min":1.4279999999999999},"min":{"1min":0.94699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7230000000000001,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.556},{"interface":"front","average":{"1min":1.198,"5min":1.3380000000000001,"15min":1.3620000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.573,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.5149999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3839999999999999,"15min":1.3979999999999999},"min":{"1min":0.83199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.694,"5min":3.052,"15min":3.052},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4379999999999999,"15min":1.4510000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9079999999999999,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":1.327}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1314816,"data_stored":882714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83299999999999996,"15min":0.84099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3999999999999999,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3420000000000001,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":0.83999999999999997}]},{"osd":2,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.56999999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.165,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":1.137},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0600000000000001,"15min":1.0860000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.5089999999999999,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":1.3200000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.96999999999999997,"15min":0.98199999999999998},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":1.296,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.661,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.397}]},{"osd":4,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1779999999999999,"15min":1.1890000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.577,"5min":2.738,"15min":2.738},"last":1.53},{"interface":"front","average":{"1min":1.171,"5min":1.3300000000000001,"15min":1.3560000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.528,"5min":3.875,"15min":3.875},"last":1.454}]},{"osd":5,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2589999999999999,"15min":1.2729999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.625,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.214},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2749999999999999,"15min":1.2909999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5680000000000001,"5min":2.54,"15min":2.54},"last":1.268}]},{"osd":6,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.698,"5min":2.117,"15min":2.117},"last":1.732},{"interface":"front","average":{"1min":1.282,"5min":1.4179999999999999,"15min":1.4399999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7230000000000001,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":1.5860000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:54:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3779999999999999,"15min":1.3859999999999999},"min":{"1min":0.84699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7629999999999999,"5min":2.931,"15min":2.931},"last":1.806},{"interface":"front","average":{"1min":1.252,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6399999999999999,"5min":4.548,"15min":4.548},"last":1.639}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:55:08.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:55:08.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.567+0000 7fe3dd41e640 1 Processor -- start 2023-12-12T09:55:08.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.567+0000 7fe3dd41e640 1 -- start start 2023-12-12T09:55:08.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.568+0000 7fe3dd41e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3d8072180 0x7fe3d80b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.568+0000 7fe3dd41e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.568+0000 7fe3dd41e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d80b3bb0 con 0x7fe3d8143e00 2023-12-12T09:55:08.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.568+0000 7fe3dd41e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d81441e0 con 0x7fe3d8072180 2023-12-12T09:55:08.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.569+0000 7fe3dd41e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d8072680 con 0x7fe3d806c410 2023-12-12T09:55:08.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.569+0000 7fe3d77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.569+0000 7fe3d67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3d8072180 0x7fe3d80b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.569+0000 7fe3d67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3d8072180 0x7fe3d80b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:51194/0 (socket says 172.21.15.72:51194) 2023-12-12T09:55:08.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.569+0000 7fe3d67fc640 1 -- 172.21.15.72:0/3481137195 learned_addr learned my addr 172.21.15.72:0/3481137195 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:08.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.570+0000 7fe3d77fe640 1 -- 172.21.15.72:0/3481137195 >> v2:172.21.15.113:3300/0 conn(0x7fe3d8072180 msgr2=0x7fe3d80b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:08.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.570+0000 7fe3d77fe640 1 --2- 172.21.15.72:0/3481137195 >> v2:172.21.15.113:3300/0 conn(0x7fe3d8072180 0x7fe3d80b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.570+0000 7fe3d77fe640 1 -- 172.21.15.72:0/3481137195 >> v1:172.21.15.113:6789/0 conn(0x7fe3d8143e00 legacy=0x7fe3d80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.570+0000 7fe3d77fe640 1 -- 172.21.15.72:0/3481137195 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80b1e30 con 0x7fe3d806c410 2023-12-12T09:55:08.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.570+0000 7fe3d67fc640 1 --2- 172.21.15.72:0/3481137195 >> v2:172.21.15.113:3300/0 conn(0x7fe3d8072180 0x7fe3d80b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:55:08.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.571+0000 7fe3d77fe640 1 --2- 172.21.15.72:0/3481137195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d815ecb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009d90 tx=0x7fe3d002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e29a87d29435dc8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:08.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.572+0000 7fe3d5ffb640 1 -- 172.21.15.72:0/3481137195 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d002cc60 con 0x7fe3d806c410 2023-12-12T09:55:08.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.572+0000 7fe3d5ffb640 1 -- 172.21.15.72:0/3481137195 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d002cdc0 con 0x7fe3d806c410 2023-12-12T09:55:08.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.572+0000 7fe3d5ffb640 1 -- 172.21.15.72:0/3481137195 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0034660 con 0x7fe3d806c410 2023-12-12T09:55:08.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.573+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3481137195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 msgr2=0x7fe3d815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.573+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3481137195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d815ecb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009d90 tx=0x7fe3d002beb0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.573+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3481137195 shutdown_connections 2023-12-12T09:55:08.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.573+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3481137195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d815ecb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.573+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3481137195 >> v2:172.21.15.113:3300/0 conn(0x7fe3d8072180 0x7fe3d80b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.573+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3481137195 >> 172.21.15.72:0/3481137195 conn(0x7fe3d80b0a90 msgr2=0x7fe3d80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:08.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.573+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3481137195 shutdown_connections 2023-12-12T09:55:08.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.573+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3481137195 wait complete. 2023-12-12T09:55:08.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.575+0000 7fe3dd41e640 1 Processor -- start 2023-12-12T09:55:08.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.575+0000 7fe3dd41e640 1 -- start start 2023-12-12T09:55:08.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3dd41e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d81f0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3dd41e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3d8072180 0x7fe3d815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3dd41e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3d8143e00 0x7fe3d815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3dd41e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d81f0a60 con 0x7fe3d8143e00 2023-12-12T09:55:08.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3dd41e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d813e300 con 0x7fe3d8072180 2023-12-12T09:55:08.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3dd41e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d813e5e0 con 0x7fe3d806c410 2023-12-12T09:55:08.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3d77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3d8143e00 0x7fe3d815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3d77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3d8143e00 0x7fe3d815bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36900/0 (socket says 172.21.15.72:36900) 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3d77fe640 1 -- 172.21.15.72:0/3363546191 learned_addr learned my addr 172.21.15.72:0/3363546191 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.576+0000 7fe3d6ffd640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d81f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3d67fc640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3d8072180 0x7fe3d815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3d77fe640 1 -- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 msgr2=0x7fe3d81f0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3d77fe640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d81f0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3d77fe640 1 -- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3d8072180 msgr2=0x7fe3d815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3d77fe640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3d8072180 0x7fe3d815b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3d77fe640 1 -- 172.21.15.72:0/3363546191 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0094d0 con 0x7fe3d8143e00 2023-12-12T09:55:08.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3d67fc640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3d8072180 0x7fe3d815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:08.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3d77fe640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3d8143e00 0x7fe3d815bd60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009d90 tx=0x7fe3d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:08.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3bf7fe640 1 -- 172.21.15.72:0/3363546191 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0040650 con 0x7fe3d8143e00 2023-12-12T09:55:08.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3bf7fe640 1 -- 172.21.15.72:0/3363546191 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d00407b0 con 0x7fe3d8143e00 2023-12-12T09:55:08.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3bf7fe640 1 -- 172.21.15.72:0/3363546191 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0040b60 con 0x7fe3d8143e00 2023-12-12T09:55:08.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.577+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d0009a70 con 0x7fe3d8143e00 2023-12-12T09:55:08.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.578+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d813eba0 con 0x7fe3d8143e00 2023-12-12T09:55:08.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.579+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d80745d0 con 0x7fe3d8143e00 2023-12-12T09:55:08.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.580+0000 7fe3bf7fe640 1 -- 172.21.15.72:0/3363546191 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3d0040cc0 con 0x7fe3d8143e00 2023-12-12T09:55:08.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.581+0000 7fe3bf7fe640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3c406ae40 0x7fe3c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:08.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.581+0000 7fe3bf7fe640 1 -- 172.21.15.72:0/3363546191 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe3d00ac1e0 con 0x7fe3d8143e00 2023-12-12T09:55:08.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.581+0000 7fe3d6ffd640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3c406ae40 0x7fe3c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:08.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.582+0000 7fe3d6ffd640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3c406ae40 0x7fe3c406d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8000f60 tx=0x7fe3c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:08.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.585+0000 7fe3bf7fe640 1 -- 172.21.15.72:0/3363546191 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3d0077440 con 0x7fe3d8143e00 2023-12-12T09:55:08.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.768+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3d806e020 con 0x7fe3d8143e00 2023-12-12T09:55:08.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.769+0000 7fe3bf7fe640 1 -- 172.21.15.72:0/3363546191 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+13193 (secure 0 0 0) 0x7fe3d0009450 con 0x7fe3d8143e00 2023-12-12T09:55:08.772 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:55:08.772 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":56,"fsid":"bd07f350-42f2-4d45-9a3d-13364a8566fe","created":"2023-12-12T09:51:48.397420+0000","modified":"2023-12-12T09:55:03.528359+0000","last_up_change":"2023-12-12T09:52:00.357249+0000","last_in_change":"2023-12-12T09:51:51.965427+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:01.617276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"be73d3b6-8cce-44e5-b7e7-662e308d68e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6825","nonce":2303781576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6827","nonce":2303781576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6831","nonce":2303781576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":2303781576},{"type":"v1","addr":"172.21.15.72:6829","nonce":2303781576}]},"public_addr":"172.21.15.72:6825/2303781576","cluster_addr":"172.21.15.72:6827/2303781576","heartbeat_back_addr":"172.21.15.72:6831/2303781576","heartbeat_front_addr":"172.21.15.72:6829/2303781576","state":["exists","up"]},{"osd":1,"uuid":"ac551d10-c0fd-4277-ae22-e8044ec35a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6801","nonce":1034360867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6803","nonce":1034360867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6807","nonce":1034360867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1034360867},{"type":"v1","addr":"172.21.15.72:6805","nonce":1034360867}]},"public_addr":"172.21.15.72:6801/1034360867","cluster_addr":"172.21.15.72:6803/1034360867","heartbeat_back_addr":"172.21.15.72:6807/1034360867","heartbeat_front_addr":"172.21.15.72:6805/1034360867","state":["exists","up"]},{"osd":2,"uuid":"633547c3-99bc-4bf2-b32b-d1a4dc2944df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6817","nonce":323046314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6819","nonce":323046314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6824","nonce":323046314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":323046314},{"type":"v1","addr":"172.21.15.72:6821","nonce":323046314}]},"public_addr":"172.21.15.72:6817/323046314","cluster_addr":"172.21.15.72:6819/323046314","heartbeat_back_addr":"172.21.15.72:6824/323046314","heartbeat_front_addr":"172.21.15.72:6821/323046314","state":["exists","up"]},{"osd":3,"uuid":"37c5f8e2-81b2-4b8f-8cb3-5c30c8f5b985","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6809","nonce":3190295309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6811","nonce":3190295309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6815","nonce":3190295309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3190295309},{"type":"v1","addr":"172.21.15.72:6813","nonce":3190295309}]},"public_addr":"172.21.15.72:6809/3190295309","cluster_addr":"172.21.15.72:6811/3190295309","heartbeat_back_addr":"172.21.15.72:6815/3190295309","heartbeat_front_addr":"172.21.15.72:6813/3190295309","state":["exists","up"]},{"osd":4,"uuid":"426889ad-4d63-416f-aeb4-71d485e31727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6801","nonce":1025641669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6803","nonce":1025641669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6807","nonce":1025641669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1025641669},{"type":"v1","addr":"172.21.15.113:6805","nonce":1025641669}]},"public_addr":"172.21.15.113:6801/1025641669","cluster_addr":"172.21.15.113:6803/1025641669","heartbeat_back_addr":"172.21.15.113:6807/1025641669","heartbeat_front_addr":"172.21.15.113:6805/1025641669","state":["exists","up"]},{"osd":5,"uuid":"c3ee9923-a20a-4b0b-8ed1-b27acfd5582d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6809","nonce":1716073726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6811","nonce":1716073726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6815","nonce":1716073726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1716073726},{"type":"v1","addr":"172.21.15.113:6813","nonce":1716073726}]},"public_addr":"172.21.15.113:6809/1716073726","cluster_addr":"172.21.15.113:6811/1716073726","heartbeat_back_addr":"172.21.15.113:6815/1716073726","heartbeat_front_addr":"172.21.15.113:6813/1716073726","state":["exists","up"]},{"osd":6,"uuid":"5a95b4bf-c94b-4246-ab6d-ac5650b885f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6817","nonce":3330758472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6819","nonce":3330758472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6823","nonce":3330758472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3330758472},{"type":"v1","addr":"172.21.15.113:6821","nonce":3330758472}]},"public_addr":"172.21.15.113:6817/3330758472","cluster_addr":"172.21.15.113:6819/3330758472","heartbeat_back_addr":"172.21.15.113:6823/3330758472","heartbeat_front_addr":"172.21.15.113:6821/3330758472","state":["exists","up"]},{"osd":7,"uuid":"1ada7456-408c-413b-a457-a982a576b8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6825","nonce":1904806807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6827","nonce":1904806807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6831","nonce":1904806807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":1904806807},{"type":"v1","addr":"172.21.15.113:6829","nonce":1904806807}]},"public_addr":"172.21.15.113:6825/1904806807","cluster_addr":"172.21.15.113:6827/1904806807","heartbeat_back_addr":"172.21.15.113:6831/1904806807","heartbeat_front_addr":"172.21.15.113:6829/1904806807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.361388+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.316578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.283311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.067898+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.035742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.063938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.263287+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.110160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6834/4181915308":"2023-12-13T09:55:00.362821+0000","172.21.15.113:6832/4181915308":"2023-12-13T09:55:00.362821+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:55:08.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3c406ae40 msgr2=0x7fe3c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3c406ae40 0x7fe3c406d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8000f60 tx=0x7fe3c8009040 comp rx=0 tx=0).stop 2023-12-12T09:55:08.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3d8143e00 msgr2=0x7fe3d815bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:08.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3d8143e00 0x7fe3d815bd60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009d90 tx=0x7fe3d0004a40 comp rx=0 tx=0).stop 2023-12-12T09:55:08.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 shutdown_connections 2023-12-12T09:55:08.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fe3c406ae40 0x7fe3c406d300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3d8143e00 0x7fe3d815bd60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3d8072180 0x7fe3d815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 --2- 172.21.15.72:0/3363546191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3d806c410 0x7fe3d81f0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:08.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 >> 172.21.15.72:0/3363546191 conn(0x7fe3d80b0a90 msgr2=0x7fe3d80cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:08.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 shutdown_connections 2023-12-12T09:55:08.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:08.772+0000 7fe3dd41e640 1 -- 172.21.15.72:0/3363546191 wait complete. 2023-12-12T09:55:09.793 INFO:tasks.ceph:Scrubbing osd.0 2023-12-12T09:55:09.793 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-12T09:55:10.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.019+0000 7f13afb41640 1 Processor -- start 2023-12-12T09:55:10.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.020+0000 7f13afb41640 1 -- start start 2023-12-12T09:55:10.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.021+0000 7f13afb41640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f13a81448a0 0x7f13a806c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.021+0000 7f13afb41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 0x7f13a815f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.021+0000 7f13afb41640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a80b3c50 con 0x7f13a80724a0 2023-12-12T09:55:10.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.021+0000 7f13afb41640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a80b3960 con 0x7f13a81448a0 2023-12-12T09:55:10.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.021+0000 7f13afb41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a815f550 con 0x7f13a815ec10 2023-12-12T09:55:10.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ad0dc640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f13a80724a0 0x7f13a80b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43104/0 (socket says 172.21.15.72:43104) 2023-12-12T09:55:10.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ad0dc640 1 -- 172.21.15.72:0/2267060822 learned_addr learned my addr 172.21.15.72:0/2267060822 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:10.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720067164 0 0) 0x7f13a80b3c50 con 0x7f13a80724a0 2023-12-12T09:55:10.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1394003520 con 0x7f13a80724a0 2023-12-12T09:55:10.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ad8dd640 1 --2- 172.21.15.72:0/2267060822 >> v2:172.21.15.113:3300/0 conn(0x7f13a81448a0 0x7f13a806c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ae0de640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 0x7f13a815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197215660 0 0) 0x7f1394003520 con 0x7f13a80724a0 2023-12-12T09:55:10.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 msgr2=0x7f13a815f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:10.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 0x7f13a815f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 >> v2:172.21.15.113:3300/0 conn(0x7f13a81448a0 msgr2=0x7f13a806c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:10.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 --2- 172.21.15.72:0/2267060822 >> v2:172.21.15.113:3300/0 conn(0x7f13a81448a0 0x7f13a806c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a8071140 con 0x7f13a80724a0 2023-12-12T09:55:10.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f1398001670 con 0x7f13a80724a0 2023-12-12T09:55:10.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.022+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 >> v1:172.21.15.72:6789/0 conn(0x7f13a80724a0 legacy=0x7f13a80b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ac8db640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1394003970 0x7f1394003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ac8db640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 0x7f13940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ac8db640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1394004d00 0x7f1394005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80b3960 con 0x7f1394004d00 2023-12-12T09:55:10.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a815f550 con 0x7f1394003970 2023-12-12T09:55:10.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139400ba30 con 0x7f13a815ec10 2023-12-12T09:55:10.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ae0de640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1394004d00 0x7f1394005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ad8dd640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1394003970 0x7f1394003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ae0de640 1 -- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 msgr2=0x7f13940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:10.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ad0dc640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 0x7f13940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ae0de640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 0x7f13940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ae0de640 1 -- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1394003970 msgr2=0x7f1394003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:10.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ae0de640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1394003970 0x7f1394003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ae0de640 1 -- 172.21.15.72:0/2267060822 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394003520 con 0x7f1394004d00 2023-12-12T09:55:10.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.023+0000 7f13ad0dc640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815ec10 0x7f13940047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:10.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.024+0000 7f13ae0de640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1394004d00 0x7f1394005180 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f13a0007390 tx=0x7f13940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6964511e4c838105 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.025+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a002ccb0 con 0x7f1394004d00 2023-12-12T09:55:10.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.025+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a002ce10 con 0x7f1394004d00 2023-12-12T09:55:10.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.025+0000 7f13ac8db640 1 -- 172.21.15.72:0/2267060822 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0034660 con 0x7f1394004d00 2023-12-12T09:55:10.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.025+0000 7f13afb41640 1 -- 172.21.15.72:0/2267060822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1394004d00 msgr2=0x7f1394005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.025+0000 7f13afb41640 1 --2- 172.21.15.72:0/2267060822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1394004d00 0x7f1394005180 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f13a0007390 tx=0x7f13940057f0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.025+0000 7f13ae0de640 1 -- 172.21.15.72:0/2267060822 reap_dead start 2023-12-12T09:55:10.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.025+0000 7f13afb41640 1 -- 172.21.15.72:0/2267060822 shutdown_connections 2023-12-12T09:55:10.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.025+0000 7f13afb41640 1 -- 172.21.15.72:0/2267060822 >> 172.21.15.72:0/2267060822 conn(0x7f13a80b0a90 msgr2=0x7f13a80b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:10.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.026+0000 7f13afb41640 1 -- 172.21.15.72:0/2267060822 shutdown_connections 2023-12-12T09:55:10.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.026+0000 7f13afb41640 1 -- 172.21.15.72:0/2267060822 wait complete. 2023-12-12T09:55:10.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.027+0000 7f13afb41640 1 Processor -- start 2023-12-12T09:55:10.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.028+0000 7f13afb41640 1 -- start start 2023-12-12T09:55:10.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.028+0000 7f13afb41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1394004d00 0x7f13a80cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13afb41640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a813f5e0 0x7f13a806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13afb41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815a460 0x7f13a815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13afb41640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a813fd50 con 0x7f13a813f5e0 2023-12-12T09:55:10.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13afb41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a813ea60 con 0x7f1394004d00 2023-12-12T09:55:10.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13afb41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a813dca0 con 0x7f13a815a460 2023-12-12T09:55:10.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad0dc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a813f5e0 0x7f13a806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad0dc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a813f5e0 0x7f13a806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36916/0 (socket says 172.21.15.72:36916) 2023-12-12T09:55:10.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad0dc640 1 -- 172.21.15.72:0/2715648478 learned_addr learned my addr 172.21.15.72:0/2715648478 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:10.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ae0de640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815a460 0x7f13a815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad0dc640 1 -- 172.21.15.72:0/2715648478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815a460 msgr2=0x7f13a815a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad0dc640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13a815a460 0x7f13a815a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad8dd640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1394004d00 0x7f13a80cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad0dc640 1 -- 172.21.15.72:0/2715648478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1394004d00 msgr2=0x7f13a80cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad0dc640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1394004d00 0x7f13a80cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.036 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.029+0000 7f13ad0dc640 1 -- 172.21.15.72:0/2715648478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a813df80 con 0x7f13a813f5e0 2023-12-12T09:55:10.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.030+0000 7f13ad0dc640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a813f5e0 0x7f13a806d6f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1398000f60 tx=0x7f139800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.030+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398007560 con 0x7f13a813f5e0 2023-12-12T09:55:10.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.030+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13980076c0 con 0x7f13a813f5e0 2023-12-12T09:55:10.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.030+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a813e260 con 0x7f13a813f5e0 2023-12-12T09:55:10.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.030+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980049e0 con 0x7f13a813f5e0 2023-12-12T09:55:10.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.030+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a813fa60 con 0x7f13a813f5e0 2023-12-12T09:55:10.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.031+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13a80b24a0 con 0x7f13a813f5e0 2023-12-12T09:55:10.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.032+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1398004b40 con 0x7f13a813f5e0 2023-12-12T09:55:10.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.033+0000 7f139e7fc640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138c06ae00 0x7f138c06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.033+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f13980892f0 con 0x7f13a813f5e0 2023-12-12T09:55:10.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.033+0000 7f139e7fc640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7f138c071100 0x7f138c073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.033+0000 7f13ad8dd640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138c06ae00 0x7f138c06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.033+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 --> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f138c073c10 con 0x7f138c071100 2023-12-12T09:55:10.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.033+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f139808b8d0 con 0x7f13a813f5e0 2023-12-12T09:55:10.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.034+0000 7f13ae0de640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7f138c071100 0x7f138c073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.034+0000 7f13ad8dd640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138c06ae00 0x7f138c06d2c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f13a4006fd0 tx=0x7f13a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.034+0000 7f13ae0de640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7f138c071100 0x7f138c073540 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.035+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 <== osd.0 v2:172.21.15.72:6823/2303781576 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f138c073c10 con 0x7f138c071100 2023-12-12T09:55:10.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.049+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 --> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13a8071700 con 0x7f138c071100 2023-12-12T09:55:10.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.050+0000 7f139e7fc640 1 -- 172.21.15.72:0/2715648478 <== osd.0 v2:172.21.15.72:6823/2303781576 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f13a002cbd0 con 0x7f138c071100 2023-12-12T09:55:10.052 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:55:10.052 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-12T09:55:10.052 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:55:10.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.050+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7f138c071100 msgr2=0x7f138c073540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.050+0000 7f13afb41640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6823/2303781576,v1:172.21.15.72:6825/2303781576] conn(0x7f138c071100 0x7f138c073540 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.050+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138c06ae00 msgr2=0x7f138c06d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.050+0000 7f13afb41640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f138c06ae00 0x7f138c06d2c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f13a4006fd0 tx=0x7f13a4008040 comp rx=0 tx=0).stop 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.051+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a813f5e0 msgr2=0x7f13a806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.051+0000 7f13afb41640 1 --2- 172.21.15.72:0/2715648478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13a813f5e0 0x7f13a806d6f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1398000f60 tx=0x7f139800df70 comp rx=0 tx=0).stop 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.051+0000 7f13ae0de640 1 -- 172.21.15.72:0/2715648478 reap_dead start 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.051+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 shutdown_connections 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.051+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 >> 172.21.15.72:0/2715648478 conn(0x7f13a80b0a90 msgr2=0x7f13a80ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.051+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 shutdown_connections 2023-12-12T09:55:10.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.051+0000 7f13afb41640 1 -- 172.21.15.72:0/2715648478 wait complete. 2023-12-12T09:55:10.069 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-12T09:55:10.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.279+0000 7f3c66e7e640 1 Processor -- start 2023-12-12T09:55:10.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.279+0000 7f3c66e7e640 1 -- start start 2023-12-12T09:55:10.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.281+0000 7f3c66e7e640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3c6015e5d0 0x7f3c6006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.281+0000 7f3c66e7e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 0x7f3c60070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.281+0000 7f3c66e7e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c60071f90 con 0x7f3c6006c330 2023-12-12T09:55:10.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.281+0000 7f3c66e7e640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c60072270 con 0x7f3c6015e5d0 2023-12-12T09:55:10.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.281+0000 7f3c66e7e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c60070070 con 0x7f3c600ae200 2023-12-12T09:55:10.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.281+0000 7f3c65e7c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3c6015e5d0 0x7f3c6006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.281+0000 7f3c65e7c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3c6015e5d0 0x7f3c6006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36920/0 (socket says 172.21.15.72:36920) 2023-12-12T09:55:10.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.281+0000 7f3c65e7c640 1 -- 172.21.15.72:0/2221100595 learned_addr learned my addr 172.21.15.72:0/2221100595 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:10.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.282+0000 7f3c6567b640 1 --2- 172.21.15.72:0/2221100595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 0x7f3c60070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.282+0000 7f3c65e7c640 1 -- 172.21.15.72:0/2221100595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 msgr2=0x7f3c60070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:10.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.282+0000 7f3c65e7c640 1 --2- 172.21.15.72:0/2221100595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 0x7f3c60070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.282+0000 7f3c65e7c640 1 -- 172.21.15.72:0/2221100595 >> v1:172.21.15.113:6789/0 conn(0x7f3c6006c330 legacy=0x7f3c600cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:10.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.282+0000 7f3c65e7c640 1 -- 172.21.15.72:0/2221100595 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c60070350 con 0x7f3c6015e5d0 2023-12-12T09:55:10.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.283+0000 7f3c65e7c640 1 --2- 172.21.15.72:0/2221100595 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c6006bdf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009870 tx=0x7f3c5c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52123f9b100cfc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.284+0000 7f3c64e7a640 1 -- 172.21.15.72:0/2221100595 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0317c0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.284+0000 7f3c64e7a640 1 -- 172.21.15.72:0/2221100595 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c031920 con 0x7f3c6015e5d0 2023-12-12T09:55:10.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.284+0000 7f3c64e7a640 1 -- 172.21.15.72:0/2221100595 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c031cb0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.284+0000 7f3c66e7e640 1 -- 172.21.15.72:0/2221100595 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 msgr2=0x7f3c6006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.284+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/2221100595 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c6006bdf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009870 tx=0x7f3c5c02b860 comp rx=0 tx=0).stop 2023-12-12T09:55:10.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.285+0000 7f3c66e7e640 1 -- 172.21.15.72:0/2221100595 shutdown_connections 2023-12-12T09:55:10.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.285+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/2221100595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 0x7f3c60070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.285+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/2221100595 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c6006bdf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.285+0000 7f3c66e7e640 1 -- 172.21.15.72:0/2221100595 >> 172.21.15.72:0/2221100595 conn(0x7f3c600b0860 msgr2=0x7f3c600b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:10.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.285+0000 7f3c66e7e640 1 -- 172.21.15.72:0/2221100595 shutdown_connections 2023-12-12T09:55:10.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.285+0000 7f3c66e7e640 1 -- 172.21.15.72:0/2221100595 wait complete. 2023-12-12T09:55:10.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.286+0000 7f3c66e7e640 1 Processor -- start 2023-12-12T09:55:10.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.287+0000 7f3c66e7e640 1 -- start start 2023-12-12T09:55:10.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c66e7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c6006c330 0x7f3c60159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c66e7e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 0x7f3c60150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c66e7e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c60147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c66e7e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c60159cc0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c66e7e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c601512f0 con 0x7f3c6006c330 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c66e7e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c60147b80 con 0x7f3c600ae200 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6667d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c60147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6667d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c60147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36926/0 (socket says 172.21.15.72:36926) 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6667d640 1 -- 172.21.15.72:0/957217314 learned_addr learned my addr 172.21.15.72:0/957217314 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c65e7c640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c6006c330 0x7f3c60159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6567b640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 0x7f3c60150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6667d640 1 -- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 msgr2=0x7f3c60150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6667d640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 0x7f3c60150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6667d640 1 -- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c6006c330 msgr2=0x7f3c60159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6667d640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c6006c330 0x7f3c60159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c6667d640 1 -- 172.21.15.72:0/957217314 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c0094d0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.288+0000 7f3c65e7c640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c6006c330 0x7f3c60159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:10.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.289+0000 7f3c6667d640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c60147640 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c7b0 tx=0x7f3c5800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.289+0000 7f3c567fc640 1 -- 172.21.15.72:0/957217314 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58004b60 con 0x7f3c6015e5d0 2023-12-12T09:55:10.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.289+0000 7f3c567fc640 1 -- 172.21.15.72:0/957217314 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58004cc0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.289+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c60151500 con 0x7f3c6015e5d0 2023-12-12T09:55:10.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.289+0000 7f3c567fc640 1 -- 172.21.15.72:0/957217314 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5800aa90 con 0x7f3c6015e5d0 2023-12-12T09:55:10.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.289+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c60151ac0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.290+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c60074450 con 0x7f3c6015e5d0 2023-12-12T09:55:10.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.291+0000 7f3c567fc640 1 -- 172.21.15.72:0/957217314 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c5800abf0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.292+0000 7f3c567fc640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.292+0000 7f3c567fc640 1 -- 172.21.15.72:0/957217314 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3c580896e0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.292+0000 7f3c65e7c640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.293+0000 7f3c65e7c640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3c4406ae40 0x7f3c4406d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c036000 tx=0x7f3c5c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.296+0000 7f3c567fc640 1 -- 172.21.15.72:0/957217314 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c580548b0 con 0x7f3c6015e5d0 2023-12-12T09:55:10.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.486+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c60071c20 con 0x7f3c4406ae40 2023-12-12T09:55:10.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.487+0000 7f3c567fc640 1 -- 172.21.15.72:0/957217314 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3c5c031b10 con 0x7f3c4406ae40 2023-12-12T09:55:10.489 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 0 to deep-scrub 2023-12-12T09:55:10.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3c4406ae40 msgr2=0x7f3c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3c4406ae40 0x7f3c4406d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c036000 tx=0x7f3c5c03d040 comp rx=0 tx=0).stop 2023-12-12T09:55:10.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 msgr2=0x7f3c60147640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c60147640 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c7b0 tx=0x7f3c5800cc70 comp rx=0 tx=0).stop 2023-12-12T09:55:10.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 shutdown_connections 2023-12-12T09:55:10.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c6015e5d0 0x7f3c60147640 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c600ae200 0x7f3c60150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 --2- 172.21.15.72:0/957217314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c6006c330 0x7f3c60159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.489+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 >> 172.21.15.72:0/957217314 conn(0x7f3c600b0860 msgr2=0x7f3c600ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:10.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.490+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 shutdown_connections 2023-12-12T09:55:10.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.490+0000 7f3c66e7e640 1 -- 172.21.15.72:0/957217314 wait complete. 2023-12-12T09:55:10.511 INFO:tasks.ceph:Scrubbing osd.1 2023-12-12T09:55:10.511 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-12T09:55:10.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.721+0000 7f6765a7c640 1 Processor -- start 2023-12-12T09:55:10.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.721+0000 7f6765a7c640 1 -- start start 2023-12-12T09:55:10.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.722+0000 7f6765a7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676006c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.722+0000 7f6765a7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67600724a0 0x7f67600b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.722+0000 7f6765a7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67600b3c50 con 0x7f67600b22d0 2023-12-12T09:55:10.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.722+0000 7f6765a7c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67600b3960 con 0x7f67600724a0 2023-12-12T09:55:10.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.722+0000 7f6765a7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6760071140 con 0x7f67601448a0 2023-12-12T09:55:10.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.723+0000 7f675f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676006c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.723+0000 7f6757fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67600724a0 0x7f67600b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.723+0000 7f675f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676006c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:36538/0 (socket says 172.21.15.72:36538) 2023-12-12T09:55:10.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.723+0000 7f675f7fe640 1 -- 172.21.15.72:0/4278174197 learned_addr learned my addr 172.21.15.72:0/4278174197 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:10.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.724+0000 7f675f7fe640 1 -- 172.21.15.72:0/4278174197 >> v2:172.21.15.113:3300/0 conn(0x7f67600724a0 msgr2=0x7f67600b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:10.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.724+0000 7f675f7fe640 1 --2- 172.21.15.72:0/4278174197 >> v2:172.21.15.113:3300/0 conn(0x7f67600724a0 0x7f67600b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.724+0000 7f675f7fe640 1 -- 172.21.15.72:0/4278174197 >> v1:172.21.15.113:6789/0 conn(0x7f67600b22d0 legacy=0x7f676015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.724+0000 7f675f7fe640 1 -- 172.21.15.72:0/4278174197 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6760071420 con 0x7f67601448a0 2023-12-12T09:55:10.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.724+0000 7f6757fff640 1 --2- 172.21.15.72:0/4278174197 >> v2:172.21.15.113:3300/0 conn(0x7f67600724a0 0x7f67600b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:55:10.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.725+0000 7f675f7fe640 1 --2- 172.21.15.72:0/4278174197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676006c430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6750009870 tx=0x7f675002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ebb266926caf53 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f675effd640 1 -- 172.21.15.72:0/4278174197 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750030840 con 0x7f67601448a0 2023-12-12T09:55:10.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f675effd640 1 -- 172.21.15.72:0/4278174197 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67500309a0 con 0x7f67601448a0 2023-12-12T09:55:10.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f675effd640 1 -- 172.21.15.72:0/4278174197 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750030d30 con 0x7f67601448a0 2023-12-12T09:55:10.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f6765a7c640 1 -- 172.21.15.72:0/4278174197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 msgr2=0x7f676006c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f6765a7c640 1 --2- 172.21.15.72:0/4278174197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676006c430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6750009870 tx=0x7f675002b980 comp rx=0 tx=0).stop 2023-12-12T09:55:10.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f6765a7c640 1 -- 172.21.15.72:0/4278174197 shutdown_connections 2023-12-12T09:55:10.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f6765a7c640 1 --2- 172.21.15.72:0/4278174197 >> v2:172.21.15.113:3300/0 conn(0x7f67600724a0 0x7f67600b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f6765a7c640 1 --2- 172.21.15.72:0/4278174197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676006c430 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f6765a7c640 1 -- 172.21.15.72:0/4278174197 >> 172.21.15.72:0/4278174197 conn(0x7f67600b0a90 msgr2=0x7f67600b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:10.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.726+0000 7f6765a7c640 1 -- 172.21.15.72:0/4278174197 shutdown_connections 2023-12-12T09:55:10.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.727+0000 7f6765a7c640 1 -- 172.21.15.72:0/4278174197 wait complete. 2023-12-12T09:55:10.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.728+0000 7f6765a7c640 1 Processor -- start 2023-12-12T09:55:10.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.728+0000 7f6765a7c640 1 -- start start 2023-12-12T09:55:10.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.729+0000 7f6765a7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67600724a0 0x7f676013b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.729+0000 7f6765a7c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67600b22d0 0x7f6760159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.729+0000 7f6765a7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.729+0000 7f6765a7c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600ccb90 con 0x7f67600b22d0 2023-12-12T09:55:10.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.729+0000 7f6765a7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676013b820 con 0x7f67600724a0 2023-12-12T09:55:10.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.729+0000 7f6765a7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6760139580 con 0x7f67601448a0 2023-12-12T09:55:10.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67600b22d0 0x7f6760159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67600b22d0 0x7f6760159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36930/0 (socket says 172.21.15.72:36930) 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 -- 172.21.15.72:0/3238923455 learned_addr learned my addr 172.21.15.72:0/3238923455 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f675ffff640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f675f7fe640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67600724a0 0x7f676013b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 -- 172.21.15.72:0/3238923455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 msgr2=0x7f676015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67601448a0 0x7f676015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 -- 172.21.15.72:0/3238923455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67600724a0 msgr2=0x7f676013b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67600724a0 0x7f676013b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 -- 172.21.15.72:0/3238923455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6748009400 con 0x7f67600b22d0 2023-12-12T09:55:10.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.730+0000 7f6757fff640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67600b22d0 0x7f6760159d80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f674800ebe0 tx=0x7f674800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.731+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748009760 con 0x7f67600b22d0 2023-12-12T09:55:10.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.731+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674800cc30 con 0x7f67600b22d0 2023-12-12T09:55:10.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.731+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674800eee0 con 0x7f67600b22d0 2023-12-12T09:55:10.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.731+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67500094d0 con 0x7f67600b22d0 2023-12-12T09:55:10.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.731+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6760139b40 con 0x7f67600b22d0 2023-12-12T09:55:10.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.732+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6760144c80 con 0x7f67600b22d0 2023-12-12T09:55:10.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.733+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6748013470 con 0x7f67600b22d0 2023-12-12T09:55:10.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.734+0000 7f675cff9640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f674006ae40 0x7f674006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.734+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f67480897f0 con 0x7f67600b22d0 2023-12-12T09:55:10.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.734+0000 7f675f7fe640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f674006ae40 0x7f674006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.734+0000 7f675cff9640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f6740071140 0x7f6740073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.734+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 --> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6740073c50 con 0x7f6740071140 2023-12-12T09:55:10.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.734+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f674808bdd0 con 0x7f67600b22d0 2023-12-12T09:55:10.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.735+0000 7f675ffff640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f6740071140 0x7f6740073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.735+0000 7f675f7fe640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f674006ae40 0x7f674006d300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6750036000 tx=0x7f675003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.735+0000 7f675ffff640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f6740071140 0x7f6740073580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.736+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 <== osd.1 v2:172.21.15.72:6800/1034360867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6740073c50 con 0x7f6740071140 2023-12-12T09:55:10.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.750+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 --> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f67600cce00 con 0x7f6740071140 2023-12-12T09:55:10.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.751+0000 7f675cff9640 1 -- 172.21.15.72:0/3238923455 <== osd.1 v2:172.21.15.72:6800/1034360867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f67600cce00 con 0x7f6740071140 2023-12-12T09:55:10.753 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:55:10.753 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-12T09:55:10.753 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:55:10.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.751+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f6740071140 msgr2=0x7f6740073580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.751+0000 7f6765a7c640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6800/1034360867,v1:172.21.15.72:6801/1034360867] conn(0x7f6740071140 0x7f6740073580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.751+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f674006ae40 msgr2=0x7f674006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.751+0000 7f6765a7c640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f674006ae40 0x7f674006d300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6750036000 tx=0x7f675003b040 comp rx=0 tx=0).stop 2023-12-12T09:55:10.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.751+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67600b22d0 msgr2=0x7f6760159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.751+0000 7f6765a7c640 1 --2- 172.21.15.72:0/3238923455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67600b22d0 0x7f6760159d80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f674800ebe0 tx=0x7f674800c520 comp rx=0 tx=0).stop 2023-12-12T09:55:10.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.752+0000 7f675ffff640 1 -- 172.21.15.72:0/3238923455 reap_dead start 2023-12-12T09:55:10.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.752+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 shutdown_connections 2023-12-12T09:55:10.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.752+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 >> 172.21.15.72:0/3238923455 conn(0x7f67600b0a90 msgr2=0x7f67600719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:10.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.752+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 shutdown_connections 2023-12-12T09:55:10.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.752+0000 7f6765a7c640 1 -- 172.21.15.72:0/3238923455 wait complete. 2023-12-12T09:55:10.772 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-12T09:55:10.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.980+0000 7f7535217640 1 Processor -- start 2023-12-12T09:55:10.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.980+0000 7f7535217640 1 -- start start 2023-12-12T09:55:10.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f7535217640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7530143e00 0x7f75300b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f7535217640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7530072180 0x7f75300b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f7535217640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75300b3bb0 con 0x7f753006c410 2023-12-12T09:55:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f7535217640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75301441e0 con 0x7f7530143e00 2023-12-12T09:55:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f7535217640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7530072680 con 0x7f7530072180 2023-12-12T09:55:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f752ed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7530143e00 0x7f75300b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f752ed9d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7530143e00 0x7f75300b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36946/0 (socket says 172.21.15.72:36946) 2023-12-12T09:55:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f752ed9d640 1 -- 172.21.15.72:0/1743358013 learned_addr learned my addr 172.21.15.72:0/1743358013 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.982+0000 7f752f59e640 1 --1- 172.21.15.72:0/1743358013 >> v1:172.21.15.72:6789/0 conn(0x7f753006c410 0x7f7530070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43114/0 (socket says 172.21.15.72:43114) 2023-12-12T09:55:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.983+0000 7f7527fff640 1 --2- 172.21.15.72:0/1743358013 >> v2:172.21.15.113:3300/0 conn(0x7f7530072180 0x7f75300b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.983+0000 7f752ed9d640 1 -- 172.21.15.72:0/1743358013 >> v2:172.21.15.113:3300/0 conn(0x7f7530072180 msgr2=0x7f75300b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.983+0000 7f752ed9d640 1 --2- 172.21.15.72:0/1743358013 >> v2:172.21.15.113:3300/0 conn(0x7f7530072180 0x7f75300b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.983+0000 7f752ed9d640 1 -- 172.21.15.72:0/1743358013 >> v1:172.21.15.72:6789/0 conn(0x7f753006c410 legacy=0x7f7530070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.983+0000 7f752ed9d640 1 -- 172.21.15.72:0/1743358013 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75300b1e30 con 0x7f7530143e00 2023-12-12T09:55:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.984+0000 7f752ed9d640 1 --2- 172.21.15.72:0/1743358013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7530143e00 0x7f75300b4040 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7530067da0 tx=0x7f752002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9971883f746b64 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.984+0000 7f752e59c640 1 -- 172.21.15.72:0/1743358013 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520030860 con 0x7f7530143e00 2023-12-12T09:55:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.984+0000 7f752e59c640 1 -- 172.21.15.72:0/1743358013 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75200309c0 con 0x7f7530143e00 2023-12-12T09:55:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.984+0000 7f752e59c640 1 -- 172.21.15.72:0/1743358013 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520030d50 con 0x7f7530143e00 2023-12-12T09:55:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.985+0000 7f7535217640 1 -- 172.21.15.72:0/1743358013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7530143e00 msgr2=0x7f75300b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.985+0000 7f7535217640 1 --2- 172.21.15.72:0/1743358013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7530143e00 0x7f75300b4040 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7530067da0 tx=0x7f752002b9a0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.985+0000 7f7535217640 1 -- 172.21.15.72:0/1743358013 shutdown_connections 2023-12-12T09:55:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.985+0000 7f7535217640 1 --2- 172.21.15.72:0/1743358013 >> v2:172.21.15.113:3300/0 conn(0x7f7530072180 0x7f75300b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.985+0000 7f7535217640 1 --2- 172.21.15.72:0/1743358013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7530143e00 0x7f75300b4040 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.985+0000 7f7535217640 1 -- 172.21.15.72:0/1743358013 >> 172.21.15.72:0/1743358013 conn(0x7f75300b0a90 msgr2=0x7f75300b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.985+0000 7f7535217640 1 -- 172.21.15.72:0/1743358013 shutdown_connections 2023-12-12T09:55:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.985+0000 7f7535217640 1 -- 172.21.15.72:0/1743358013 wait complete. 2023-12-12T09:55:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.987+0000 7f7535217640 1 Processor -- start 2023-12-12T09:55:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.988+0000 7f7535217640 1 -- start start 2023-12-12T09:55:10.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f7535217640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f753006c410 0x7f7530149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f7535217640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7530072180 0x7f7530140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f7535217640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7530143e00 0x7f753014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f7535217640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530142470 con 0x7f753006c410 2023-12-12T09:55:10.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f7535217640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753014c4d0 con 0x7f7530072180 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f7535217640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753014c7b0 con 0x7f7530143e00 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f752ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f753006c410 0x7f7530149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f752ed9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f753006c410 0x7f7530149980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36960/0 (socket says 172.21.15.72:36960) 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f752ed9d640 1 -- 172.21.15.72:0/1675204958 learned_addr learned my addr 172.21.15.72:0/1675204958 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f752ed9d640 1 -- 172.21.15.72:0/1675204958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7530143e00 msgr2=0x7f753014b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f752ed9d640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7530143e00 0x7f753014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f752ed9d640 1 -- 172.21.15.72:0/1675204958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7530072180 msgr2=0x7f7530140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f752ed9d640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7530072180 0x7f7530140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.989+0000 7f752ed9d640 1 -- 172.21.15.72:0/1675204958 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75200094d0 con 0x7f753006c410 2023-12-12T09:55:10.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.990+0000 7f752ed9d640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f753006c410 0x7f7530149980 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f752002b9e0 tx=0x7f7520004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.990+0000 7f75277fe640 1 -- 172.21.15.72:0/1675204958 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520039b60 con 0x7f753006c410 2023-12-12T09:55:10.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.990+0000 7f75277fe640 1 -- 172.21.15.72:0/1675204958 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7520039ce0 con 0x7f753006c410 2023-12-12T09:55:10.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.990+0000 7f75277fe640 1 -- 172.21.15.72:0/1675204958 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520043650 con 0x7f753006c410 2023-12-12T09:55:10.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.990+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7530142100 con 0x7f753006c410 2023-12-12T09:55:10.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.990+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753015ac90 con 0x7f753006c410 2023-12-12T09:55:10.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.991+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753014c970 con 0x7f753006c410 2023-12-12T09:55:10.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.992+0000 7f75277fe640 1 -- 172.21.15.72:0/1675204958 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f752002f3b0 con 0x7f753006c410 2023-12-12T09:55:10.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.993+0000 7f75277fe640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f751006ae70 0x7f751006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:10.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.993+0000 7f75277fe640 1 -- 172.21.15.72:0/1675204958 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f75200ab6f0 con 0x7f753006c410 2023-12-12T09:55:10.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.993+0000 7f7527fff640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f751006ae70 0x7f751006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:10.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.994+0000 7f7527fff640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f751006ae70 0x7f751006d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f753006d620 tx=0x7f7518008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:10.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:10.998+0000 7f75277fe640 1 -- 172.21.15.72:0/1675204958 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7520076960 con 0x7f753006c410 2023-12-12T09:55:11.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.188+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f75300b23f0 con 0x7f751006ae70 2023-12-12T09:55:11.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.189+0000 7f75277fe640 1 -- 172.21.15.72:0/1675204958 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f751800d7b0 con 0x7f751006ae70 2023-12-12T09:55:11.191 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 1 to deep-scrub 2023-12-12T09:55:11.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f751006ae70 msgr2=0x7f751006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f751006ae70 0x7f751006d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f753006d620 tx=0x7f7518008040 comp rx=0 tx=0).stop 2023-12-12T09:55:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f753006c410 msgr2=0x7f7530149980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f753006c410 0x7f7530149980 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f752002b9e0 tx=0x7f7520004730 comp rx=0 tx=0).stop 2023-12-12T09:55:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 shutdown_connections 2023-12-12T09:55:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f751006ae70 0x7f751006d330 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7530143e00 0x7f753014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7530072180 0x7f7530140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 --2- 172.21.15.72:0/1675204958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f753006c410 0x7f7530149980 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 >> 172.21.15.72:0/1675204958 conn(0x7f75300b0a90 msgr2=0x7f75300caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 shutdown_connections 2023-12-12T09:55:11.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.192+0000 7f7535217640 1 -- 172.21.15.72:0/1675204958 wait complete. 2023-12-12T09:55:11.212 INFO:tasks.ceph:Scrubbing osd.2 2023-12-12T09:55:11.213 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-12T09:55:11.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.425+0000 7fabf03b0640 1 Processor -- start 2023-12-12T09:55:11.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.426+0000 7fabf03b0640 1 -- start start 2023-12-12T09:55:11.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.427+0000 7fabf03b0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fabe80999f0 0x7fabe8099e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.427+0000 7fabf03b0640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe806ca90 con 0x7fabe80b0940 2023-12-12T09:55:11.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.427+0000 7fabf03b0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe809a370 con 0x7fabe80b2340 2023-12-12T09:55:11.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.427+0000 7fabf03b0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe806cf20 con 0x7fabe80999f0 2023-12-12T09:55:11.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.427+0000 7fabee94d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fabe80b0940 0x7fabe80b0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43116/0 (socket says 172.21.15.72:43116) 2023-12-12T09:55:11.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.427+0000 7fabee94d640 1 -- 172.21.15.72:0/82067146 learned_addr learned my addr 172.21.15.72:0/82067146 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:11.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed94b640 1 --2- 172.21.15.72:0/82067146 >> v2:172.21.15.113:3300/0 conn(0x7fabe80999f0 0x7fabe8099e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356488240 0 0) 0x7fabe806ca90 con 0x7fabe80b0940 2023-12-12T09:55:11.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd4003520 con 0x7fabe80b0940 2023-12-12T09:55:11.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589275122 0 0) 0x7fabd4003520 con 0x7fabe80b0940 2023-12-12T09:55:11.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 >> v2:172.21.15.113:3300/0 conn(0x7fabe80999f0 msgr2=0x7fabe8099e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:11.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 --2- 172.21.15.72:0/82067146 >> v2:172.21.15.113:3300/0 conn(0x7fabe80999f0 0x7fabe8099e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 >> v1:172.21.15.113:6789/0 conn(0x7fabe80b2340 legacy=0x7fabe80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80b3890 con 0x7fabe80b0940 2023-12-12T09:55:11.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7fabe0001670 con 0x7fabe80b0940 2023-12-12T09:55:11.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 >> v1:172.21.15.72:6789/0 conn(0x7fabe80b0940 legacy=0x7fabe80b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.428+0000 7fabed14a640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd40039c0 0x7fabd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed14a640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabd40047c0 0x7fabd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed14a640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd4005180 0x7fabd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe806cf20 con 0x7fabd40047c0 2023-12-12T09:55:11.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd400c080 con 0x7fabd4005180 2023-12-12T09:55:11.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd400c240 con 0x7fabd40039c0 2023-12-12T09:55:11.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed94b640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabd40047c0 0x7fabd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed94b640 1 -- 172.21.15.72:0/82067146 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd40039c0 msgr2=0x7fabd4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:11.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed94b640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd40039c0 0x7fabd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed94b640 1 -- 172.21.15.72:0/82067146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd4005180 msgr2=0x7fabd400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:11.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed94b640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd4005180 0x7fabd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabed94b640 1 -- 172.21.15.72:0/82067146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd4003520 con 0x7fabd40047c0 2023-12-12T09:55:11.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.429+0000 7fabee94d640 1 -- 172.21.15.72:0/82067146 reap_dead start 2023-12-12T09:55:11.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.430+0000 7fabed94b640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabd40047c0 0x7fabd4004c40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009ac0 tx=0x7fabd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43748b61a3047a79 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:11.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.430+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd802c6b0 con 0x7fabd40047c0 2023-12-12T09:55:11.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.430+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd802c810 con 0x7fabd40047c0 2023-12-12T09:55:11.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.430+0000 7fabed14a640 1 -- 172.21.15.72:0/82067146 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd802cba0 con 0x7fabd40047c0 2023-12-12T09:55:11.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.431+0000 7fabf03b0640 1 -- 172.21.15.72:0/82067146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabd40047c0 msgr2=0x7fabd4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.431+0000 7fabf03b0640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabd40047c0 0x7fabd4004c40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009ac0 tx=0x7fabd40057f0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.431+0000 7fabf03b0640 1 -- 172.21.15.72:0/82067146 shutdown_connections 2023-12-12T09:55:11.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.431+0000 7fabf03b0640 1 --2- 172.21.15.72:0/82067146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabd40047c0 0x7fabd4004c40 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.431+0000 7fabf03b0640 1 -- 172.21.15.72:0/82067146 >> 172.21.15.72:0/82067146 conn(0x7fabe80b2ad0 msgr2=0x7fabe80b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:11.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.431+0000 7fabf03b0640 1 -- 172.21.15.72:0/82067146 shutdown_connections 2023-12-12T09:55:11.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.431+0000 7fabf03b0640 1 -- 172.21.15.72:0/82067146 wait complete. 2023-12-12T09:55:11.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.433+0000 7fabf03b0640 1 Processor -- start 2023-12-12T09:55:11.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.433+0000 7fabf03b0640 1 -- start start 2023-12-12T09:55:11.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabf03b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd40047c0 0x7fabe8155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabf03b0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabe81f37d0 0x7fabe80ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabf03b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe80cd1c0 0x7fabe80ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabf03b0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b4fa0 con 0x7fabe81f37d0 2023-12-12T09:55:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabf03b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b5280 con 0x7fabe80cd1c0 2023-12-12T09:55:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabf03b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b5560 con 0x7fabd40047c0 2023-12-12T09:55:11.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabed94b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabe81f37d0 0x7fabe80ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabed94b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabe81f37d0 0x7fabe80ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36978/0 (socket says 172.21.15.72:36978) 2023-12-12T09:55:11.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.434+0000 7fabed94b640 1 -- 172.21.15.72:0/3161284157 learned_addr learned my addr 172.21.15.72:0/3161284157 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:11.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabee14c640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd40047c0 0x7fabe8155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabee94d640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe80cd1c0 0x7fabe80ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabed94b640 1 -- 172.21.15.72:0/3161284157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd40047c0 msgr2=0x7fabe8155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabed94b640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd40047c0 0x7fabe8155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabed94b640 1 -- 172.21.15.72:0/3161284157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe80cd1c0 msgr2=0x7fabe80ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabed94b640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe80cd1c0 0x7fabe80ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabed94b640 1 -- 172.21.15.72:0/3161284157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80b5840 con 0x7fabe81f37d0 2023-12-12T09:55:11.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabee94d640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe80cd1c0 0x7fabe80ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:11.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabed94b640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabe81f37d0 0x7fabe80ccc80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fabd802c250 tx=0x7fabd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:11.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8003810 con 0x7fabe81f37d0 2023-12-12T09:55:11.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd8003970 con 0x7fabe81f37d0 2023-12-12T09:55:11.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8003d00 con 0x7fabe81f37d0 2023-12-12T09:55:11.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.435+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe80b5b20 con 0x7fabe81f37d0 2023-12-12T09:55:11.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.436+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe81f3c50 con 0x7fabe81f37d0 2023-12-12T09:55:11.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.436+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fabe806c440 con 0x7fabe81f37d0 2023-12-12T09:55:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.437+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fabd8034440 con 0x7fabe81f37d0 2023-12-12T09:55:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.438+0000 7fabdeffd640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fabcc06ae70 0x7fabcc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.438+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fabd80ab6f0 con 0x7fabe81f37d0 2023-12-12T09:55:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.439+0000 7fabee14c640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fabcc06ae70 0x7fabcc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.439+0000 7fabdeffd640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7fabcc071170 0x7fabcc0735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.439+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 --> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabcc073c80 con 0x7fabcc071170 2023-12-12T09:55:11.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.439+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabd80addb0 con 0x7fabe81f37d0 2023-12-12T09:55:11.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.439+0000 7fabee94d640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7fabcc071170 0x7fabcc0735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.439+0000 7fabee14c640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fabcc06ae70 0x7fabcc06d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007fd0 tx=0x7fabe4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:11.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.440+0000 7fabee94d640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7fabcc071170 0x7fabcc0735b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:11.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.440+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 <== osd.2 v2:172.21.15.72:6816/323046314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fabcc073c80 con 0x7fabcc071170 2023-12-12T09:55:11.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.455+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 --> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fabe806c160 con 0x7fabcc071170 2023-12-12T09:55:11.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.455+0000 7fabdeffd640 1 -- 172.21.15.72:0/3161284157 <== osd.2 v2:172.21.15.72:6816/323046314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fabe0001bc0 con 0x7fabcc071170 2023-12-12T09:55:11.460 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:55:11.460 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-12T09:55:11.461 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:55:11.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7fabcc071170 msgr2=0x7fabcc0735b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6816/323046314,v1:172.21.15.72:6817/323046314] conn(0x7fabcc071170 0x7fabcc0735b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fabcc06ae70 msgr2=0x7fabcc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fabcc06ae70 0x7fabcc06d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007fd0 tx=0x7fabe4009040 comp rx=0 tx=0).stop 2023-12-12T09:55:11.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabe81f37d0 msgr2=0x7fabe80ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 --2- 172.21.15.72:0/3161284157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabe81f37d0 0x7fabe80ccc80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fabd802c250 tx=0x7fabd8004730 comp rx=0 tx=0).stop 2023-12-12T09:55:11.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabee94d640 1 -- 172.21.15.72:0/3161284157 reap_dead start 2023-12-12T09:55:11.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 shutdown_connections 2023-12-12T09:55:11.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 >> 172.21.15.72:0/3161284157 conn(0x7fabe80b2ad0 msgr2=0x7fabe806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:11.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 shutdown_connections 2023-12-12T09:55:11.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.456+0000 7fabf03b0640 1 -- 172.21.15.72:0/3161284157 wait complete. 2023-12-12T09:55:11.476 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-12T09:55:11.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.684+0000 7f182f1b6640 1 Processor -- start 2023-12-12T09:55:11.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.685+0000 7f182f1b6640 1 -- start start 2023-12-12T09:55:11.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.686+0000 7f182f1b6640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f182815f260 0x7f18280ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.686+0000 7f182f1b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18281626b0 0x7f18280701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.686+0000 7f182f1b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1828070b60 con 0x7f1828070700 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.686+0000 7f182f1b6640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18280cd040 con 0x7f182815f260 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.686+0000 7f182f1b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182815f640 con 0x7f18281626b0 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.686+0000 7f182cf52640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f182815f260 0x7f18280ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.686+0000 7f182cf52640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f182815f260 0x7f18280ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36994/0 (socket says 172.21.15.72:36994) 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.686+0000 7f182cf52640 1 -- 172.21.15.72:0/1609392068 learned_addr learned my addr 172.21.15.72:0/1609392068 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.687+0000 7f182cf52640 1 -- 172.21.15.72:0/1609392068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18281626b0 msgr2=0x7f18280701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.687+0000 7f182cf52640 1 --2- 172.21.15.72:0/1609392068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18281626b0 0x7f18280701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.687+0000 7f182cf52640 1 -- 172.21.15.72:0/1609392068 >> v1:172.21.15.113:6789/0 conn(0x7f1828070700 legacy=0x7f182806c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:11.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.687+0000 7f182cf52640 1 -- 172.21.15.72:0/1609392068 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280cde40 con 0x7f182815f260 2023-12-12T09:55:11.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.688+0000 7f182cf52640 1 --2- 172.21.15.72:0/1609392068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f18280ccb00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6363eab1dbfc5a00 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:11.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.688+0000 7f18277fe640 1 -- 172.21.15.72:0/1609392068 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c031640 con 0x7f182815f260 2023-12-12T09:55:11.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.688+0000 7f18277fe640 1 -- 172.21.15.72:0/1609392068 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0317a0 con 0x7f182815f260 2023-12-12T09:55:11.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.688+0000 7f18277fe640 1 -- 172.21.15.72:0/1609392068 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c031b30 con 0x7f182815f260 2023-12-12T09:55:11.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.688+0000 7f182f1b6640 1 -- 172.21.15.72:0/1609392068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 msgr2=0x7f18280ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.688+0000 7f182f1b6640 1 --2- 172.21.15.72:0/1609392068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f18280ccb00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c02b8f0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.689+0000 7f182f1b6640 1 -- 172.21.15.72:0/1609392068 shutdown_connections 2023-12-12T09:55:11.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.689+0000 7f182f1b6640 1 --2- 172.21.15.72:0/1609392068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18281626b0 0x7f18280701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.689+0000 7f182f1b6640 1 --2- 172.21.15.72:0/1609392068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f18280ccb00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.689+0000 7f182f1b6640 1 -- 172.21.15.72:0/1609392068 >> 172.21.15.72:0/1609392068 conn(0x7f18280b0a90 msgr2=0x7f18280b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:11.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.689+0000 7f182f1b6640 1 -- 172.21.15.72:0/1609392068 shutdown_connections 2023-12-12T09:55:11.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.689+0000 7f182f1b6640 1 -- 172.21.15.72:0/1609392068 wait complete. 2023-12-12T09:55:11.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.690+0000 7f182f1b6640 1 Processor -- start 2023-12-12T09:55:11.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.691+0000 7f182f1b6640 1 -- start start 2023-12-12T09:55:11.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f182f1b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1828070700 0x7f18280acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f182f1b6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f182815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f182f1b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18281626b0 0x7f18280aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f182f1b6640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18280ad120 con 0x7f182815f260 2023-12-12T09:55:11.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f182f1b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18281f7230 con 0x7f1828070700 2023-12-12T09:55:11.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f182f1b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18281f7510 con 0x7f18281626b0 2023-12-12T09:55:11.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f1827fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f182815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f1827fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f182815f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36998/0 (socket says 172.21.15.72:36998) 2023-12-12T09:55:11.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f1827fff640 1 -- 172.21.15.72:0/2746251330 learned_addr learned my addr 172.21.15.72:0/2746251330 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:11.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f1827fff640 1 -- 172.21.15.72:0/2746251330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18281626b0 msgr2=0x7f18280aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f1827fff640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18281626b0 0x7f18280aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f182cf52640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1828070700 0x7f18280acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f1827fff640 1 -- 172.21.15.72:0/2746251330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1828070700 msgr2=0x7f18280acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f1827fff640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1828070700 0x7f18280acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.692+0000 7f1827fff640 1 -- 172.21.15.72:0/2746251330 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0094d0 con 0x7f182815f260 2023-12-12T09:55:11.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.693+0000 7f1827fff640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f182815f9f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f181400b410 tx=0x7f181400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:11.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.693+0000 7f18257fa640 1 -- 172.21.15.72:0/2746251330 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814011830 con 0x7f182815f260 2023-12-12T09:55:11.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.693+0000 7f18257fa640 1 -- 172.21.15.72:0/2746251330 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18140119b0 con 0x7f182815f260 2023-12-12T09:55:11.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.693+0000 7f18257fa640 1 -- 172.21.15.72:0/2746251330 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814011d60 con 0x7f182815f260 2023-12-12T09:55:11.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.693+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18281f77f0 con 0x7f182815f260 2023-12-12T09:55:11.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.693+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18281f7e40 con 0x7f182815f260 2023-12-12T09:55:11.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.694+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182814f060 con 0x7f182815f260 2023-12-12T09:55:11.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.695+0000 7f18257fa640 1 -- 172.21.15.72:0/2746251330 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f181401a400 con 0x7f182815f260 2023-12-12T09:55:11.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.696+0000 7f18257fa640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f180c06ae40 0x7f180c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:11.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.696+0000 7f18257fa640 1 -- 172.21.15.72:0/2746251330 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1814089540 con 0x7f182815f260 2023-12-12T09:55:11.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.696+0000 7f182cf52640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f180c06ae40 0x7f180c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:11.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.697+0000 7f182cf52640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f180c06ae40 0x7f180c06d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f181c036000 tx=0x7f181c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:11.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.700+0000 7f18257fa640 1 -- 172.21.15.72:0/2746251330 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f182814f060 con 0x7f182815f260 2023-12-12T09:55:11.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.890+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f18280cd510 con 0x7f180c06ae40 2023-12-12T09:55:11.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.890+0000 7f18257fa640 1 -- 172.21.15.72:0/2746251330 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f181c031ae0 con 0x7f180c06ae40 2023-12-12T09:55:11.893 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 2 to deep-scrub 2023-12-12T09:55:11.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f180c06ae40 msgr2=0x7f180c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f180c06ae40 0x7f180c06d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f181c036000 tx=0x7f181c03c040 comp rx=0 tx=0).stop 2023-12-12T09:55:11.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 msgr2=0x7f182815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:11.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f182815f9f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f181400b410 tx=0x7f181400b8d0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 shutdown_connections 2023-12-12T09:55:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f180c06ae40 0x7f180c06d300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18281626b0 0x7f18280aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182815f260 0x7f182815f9f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 --2- 172.21.15.72:0/2746251330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1828070700 0x7f18280acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 >> 172.21.15.72:0/2746251330 conn(0x7f18280b0a90 msgr2=0x7f18280b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 shutdown_connections 2023-12-12T09:55:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:11.893+0000 7f182f1b6640 1 -- 172.21.15.72:0/2746251330 wait complete. 2023-12-12T09:55:11.913 INFO:tasks.ceph:Scrubbing osd.3 2023-12-12T09:55:11.914 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-12T09:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.129+0000 7f0025fc9640 1 Processor -- start 2023-12-12T09:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.129+0000 7f0025fc9640 1 -- start start 2023-12-12T09:55:12.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.130+0000 7f0025fc9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00200b3770 0x7f00200cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.130+0000 7f0025fc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200ce1b0 0x7f00200cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.130+0000 7f0025fc9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002015c4b0 con 0x7f00200ce630 2023-12-12T09:55:12.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.130+0000 7f0025fc9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00200cd750 con 0x7f00200b3770 2023-12-12T09:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.130+0000 7f0025fc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00200cb700 con 0x7f00200ce1b0 2023-12-12T09:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.131+0000 7f0024fc7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00200b3770 0x7f00200cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.131+0000 7f001ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200ce1b0 0x7f00200cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.131+0000 7f0024fc7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00200b3770 0x7f00200cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.72:51324/0 (socket says 172.21.15.72:51324) 2023-12-12T09:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.131+0000 7f0024fc7640 1 -- 172.21.15.72:0/3095619532 learned_addr learned my addr 172.21.15.72:0/3095619532 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.132+0000 7f0024fc7640 1 -- 172.21.15.72:0/3095619532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200ce1b0 msgr2=0x7f00200cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.132+0000 7f0024fc7640 1 --2- 172.21.15.72:0/3095619532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200ce1b0 0x7f00200cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.132+0000 7f0024fc7640 1 -- 172.21.15.72:0/3095619532 >> v1:172.21.15.113:6789/0 conn(0x7f00200ce630 legacy=0x7f00200b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.132+0000 7f0024fc7640 1 -- 172.21.15.72:0/3095619532 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200b3b50 con 0x7f00200b3770 2023-12-12T09:55:12.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.133+0000 7f0024fc7640 1 --2- 172.21.15.72:0/3095619532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200b3770 0x7f00200cb1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0014009870 tx=0x7f001402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7d3d90beb55bbd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.134+0000 7f001f7fe640 1 -- 172.21.15.72:0/3095619532 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014030840 con 0x7f00200b3770 2023-12-12T09:55:12.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.134+0000 7f001f7fe640 1 -- 172.21.15.72:0/3095619532 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00140309a0 con 0x7f00200b3770 2023-12-12T09:55:12.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.134+0000 7f001f7fe640 1 -- 172.21.15.72:0/3095619532 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014030d30 con 0x7f00200b3770 2023-12-12T09:55:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.134+0000 7f0025fc9640 1 -- 172.21.15.72:0/3095619532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200b3770 msgr2=0x7f00200cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.134+0000 7f0025fc9640 1 --2- 172.21.15.72:0/3095619532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200b3770 0x7f00200cb1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0014009870 tx=0x7f001402b980 comp rx=0 tx=0).stop 2023-12-12T09:55:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.135+0000 7f0025fc9640 1 -- 172.21.15.72:0/3095619532 shutdown_connections 2023-12-12T09:55:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.135+0000 7f0025fc9640 1 --2- 172.21.15.72:0/3095619532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200ce1b0 0x7f00200cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.135+0000 7f0025fc9640 1 --2- 172.21.15.72:0/3095619532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200b3770 0x7f00200cb1c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.135+0000 7f0025fc9640 1 -- 172.21.15.72:0/3095619532 >> 172.21.15.72:0/3095619532 conn(0x7f002006a880 msgr2=0x7f002006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.135+0000 7f0025fc9640 1 -- 172.21.15.72:0/3095619532 shutdown_connections 2023-12-12T09:55:12.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.135+0000 7f0025fc9640 1 -- 172.21.15.72:0/3095619532 wait complete. 2023-12-12T09:55:12.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.136+0000 7f0025fc9640 1 Processor -- start 2023-12-12T09:55:12.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.137+0000 7f0025fc9640 1 -- start start 2023-12-12T09:55:12.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.137+0000 7f0025fc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200b3770 0x7f00201f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.137+0000 7f0025fc9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00200ce1b0 0x7f00200acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.137+0000 7f0025fc9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200ce630 0x7f00200abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.137+0000 7f0025fc9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002009ca60 con 0x7f00200ce1b0 2023-12-12T09:55:12.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.137+0000 7f0025fc9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0020155ad0 con 0x7f00200ce630 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.137+0000 7f0025fc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00201f6c70 con 0x7f00200b3770 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f001ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00200ce1b0 0x7f00200acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f001ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00200ce1b0 0x7f00200acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37004/0 (socket says 172.21.15.72:37004) 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f001ffff640 1 -- 172.21.15.72:0/2870038978 learned_addr learned my addr 172.21.15.72:0/2870038978 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f0024fc7640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200b3770 0x7f00201f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f00257c8640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200ce630 0x7f00200abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f001ffff640 1 -- 172.21.15.72:0/2870038978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200b3770 msgr2=0x7f00201f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f001ffff640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00200b3770 0x7f00201f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f001ffff640 1 -- 172.21.15.72:0/2870038978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200ce630 msgr2=0x7f00200abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f001ffff640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200ce630 0x7f00200abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f001ffff640 1 -- 172.21.15.72:0/2870038978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00140094d0 con 0x7f00200ce1b0 2023-12-12T09:55:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.138+0000 7f00257c8640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00200ce630 0x7f00200abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.139+0000 7f001ffff640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00200ce1b0 0x7f00200acac0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f000c00e9b0 tx=0x7f000c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.139+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c009670 con 0x7f00200ce1b0 2023-12-12T09:55:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.139+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c00c9a0 con 0x7f00200ce1b0 2023-12-12T09:55:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.139+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00cd50 con 0x7f00200ce1b0 2023-12-12T09:55:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.139+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00201f6f50 con 0x7f00200ce1b0 2023-12-12T09:55:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.139+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0020155d70 con 0x7f00200ce1b0 2023-12-12T09:55:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.140+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00200ccc90 con 0x7f00200ce1b0 2023-12-12T09:55:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.141+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f000c013470 con 0x7f00200ce1b0 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.142+0000 7f001d7fa640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f000406ae40 0x7f000406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.142+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f000c089660 con 0x7f00200ce1b0 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.142+0000 7f0024fc7640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f000406ae40 0x7f000406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.142+0000 7f001d7fa640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7f0004071140 0x7f0004073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.142+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 --> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0004073c50 con 0x7f0004071140 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.142+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f000c08bcc0 con 0x7f00200ce1b0 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.143+0000 7f00257c8640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7f0004071140 0x7f0004073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.143+0000 7f0024fc7640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f000406ae40 0x7f000406d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0014036000 tx=0x7f001403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.143+0000 7f00257c8640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7f0004071140 0x7f0004073580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.144+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 <== osd.3 v2:172.21.15.72:6808/3190295309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0004073c50 con 0x7f0004071140 2023-12-12T09:55:12.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.158+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 --> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f00200cdb00 con 0x7f0004071140 2023-12-12T09:55:12.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.159+0000 7f001d7fa640 1 -- 172.21.15.72:0/2870038978 <== osd.3 v2:172.21.15.72:6808/3190295309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f00200cdb00 con 0x7f0004071140 2023-12-12T09:55:12.161 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:55:12.161 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-12T09:55:12.161 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:55:12.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.159+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7f0004071140 msgr2=0x7f0004073580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.159+0000 7f0025fc9640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6808/3190295309,v1:172.21.15.72:6809/3190295309] conn(0x7f0004071140 0x7f0004073580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f000406ae40 msgr2=0x7f000406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f0025fc9640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f000406ae40 0x7f000406d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0014036000 tx=0x7f001403b040 comp rx=0 tx=0).stop 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00200ce1b0 msgr2=0x7f00200acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f0025fc9640 1 --2- 172.21.15.72:0/2870038978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00200ce1b0 0x7f00200acac0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f000c00e9b0 tx=0x7f000c00ee70 comp rx=0 tx=0).stop 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f00257c8640 1 -- 172.21.15.72:0/2870038978 reap_dead start 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 shutdown_connections 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 >> 172.21.15.72:0/2870038978 conn(0x7f002006a880 msgr2=0x7f00200ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 shutdown_connections 2023-12-12T09:55:12.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.160+0000 7f0025fc9640 1 -- 172.21.15.72:0/2870038978 wait complete. 2023-12-12T09:55:12.180 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-12T09:55:12.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.391+0000 7f220aa21640 1 Processor -- start 2023-12-12T09:55:12.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.392+0000 7f220aa21640 1 -- start start 2023-12-12T09:55:12.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.393+0000 7f220aa21640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f220415ada0 0x7f220406e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.393+0000 7f220aa21640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f22040cefd0 0x7f2204070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.393+0000 7f220aa21640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220406b3c0 con 0x7f22040712b0 2023-12-12T09:55:12.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.393+0000 7f220aa21640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2204071740 con 0x7f22040cefd0 2023-12-12T09:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.393+0000 7f220aa21640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22040ceb40 con 0x7f220415ada0 2023-12-12T09:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.394+0000 7f220a220640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f22040712b0 0x7f220406c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43122/0 (socket says 172.21.15.72:43122) 2023-12-12T09:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.394+0000 7f220a220640 1 -- 172.21.15.72:0/3581612434 learned_addr learned my addr 172.21.15.72:0/3581612434 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.394+0000 7f2209a1f640 1 --2- 172.21.15.72:0/3581612434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f220415ada0 0x7f220406e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.394+0000 7f220921e640 1 --2- 172.21.15.72:0/3581612434 >> v2:172.21.15.72:3300/0 conn(0x7f22040cefd0 0x7f2204070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.394+0000 7f2208a1d640 1 -- 172.21.15.72:0/3581612434 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740493691 0 0) 0x7f220406b3c0 con 0x7f22040712b0 2023-12-12T09:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.394+0000 7f2208a1d640 1 -- 172.21.15.72:0/3581612434 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f0003520 con 0x7f22040712b0 2023-12-12T09:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.395+0000 7f220921e640 1 -- 172.21.15.72:0/3581612434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f220415ada0 msgr2=0x7f220406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.395+0000 7f220921e640 1 --2- 172.21.15.72:0/3581612434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f220415ada0 0x7f220406e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.395+0000 7f220921e640 1 -- 172.21.15.72:0/3581612434 >> v1:172.21.15.72:6789/0 conn(0x7f22040712b0 legacy=0x7f220406c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.395+0000 7f220921e640 1 -- 172.21.15.72:0/3581612434 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040cdf30 con 0x7f22040cefd0 2023-12-12T09:55:12.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.396+0000 7f220921e640 1 --2- 172.21.15.72:0/3581612434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040cefd0 0x7f2204070d70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f21f80097a0 tx=0x7f21f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1603382f02aea2d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.396+0000 7f2208a1d640 1 -- 172.21.15.72:0/3581612434 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8031830 con 0x7f22040cefd0 2023-12-12T09:55:12.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.396+0000 7f2208a1d640 1 -- 172.21.15.72:0/3581612434 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f8031990 con 0x7f22040cefd0 2023-12-12T09:55:12.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.396+0000 7f2208a1d640 1 -- 172.21.15.72:0/3581612434 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8031d20 con 0x7f22040cefd0 2023-12-12T09:55:12.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.397+0000 7f220aa21640 1 -- 172.21.15.72:0/3581612434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040cefd0 msgr2=0x7f2204070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.397+0000 7f220aa21640 1 --2- 172.21.15.72:0/3581612434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040cefd0 0x7f2204070d70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f21f80097a0 tx=0x7f21f802b8d0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.397+0000 7f220aa21640 1 -- 172.21.15.72:0/3581612434 shutdown_connections 2023-12-12T09:55:12.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.397+0000 7f220aa21640 1 --2- 172.21.15.72:0/3581612434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040cefd0 0x7f2204070d70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.397+0000 7f220aa21640 1 --2- 172.21.15.72:0/3581612434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f220415ada0 0x7f220406e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.397+0000 7f220aa21640 1 -- 172.21.15.72:0/3581612434 >> 172.21.15.72:0/3581612434 conn(0x7f22040b1540 msgr2=0x7f22040b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.397+0000 7f220aa21640 1 -- 172.21.15.72:0/3581612434 shutdown_connections 2023-12-12T09:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.397+0000 7f220aa21640 1 -- 172.21.15.72:0/3581612434 wait complete. 2023-12-12T09:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.399+0000 7f220aa21640 1 Processor -- start 2023-12-12T09:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.399+0000 7f220aa21640 1 -- start start 2023-12-12T09:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f220aa21640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040712b0 0x7f2204140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f220aa21640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22040cefd0 0x7f2204146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f220aa21640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f220415ada0 0x7f2204146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f220aa21640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204150cc0 con 0x7f22040712b0 2023-12-12T09:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f220aa21640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040cb6c0 con 0x7f220415ada0 2023-12-12T09:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f220aa21640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220415c9f0 con 0x7f22040cefd0 2023-12-12T09:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f2209a1f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040712b0 0x7f2204140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f2209a1f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040712b0 0x7f2204140f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37020/0 (socket says 172.21.15.72:37020) 2023-12-12T09:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.400+0000 7f2209a1f640 1 -- 172.21.15.72:0/269433183 learned_addr learned my addr 172.21.15.72:0/269433183 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f220921e640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22040cefd0 0x7f2204146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f220a220640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f220415ada0 0x7f2204146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f2209a1f640 1 -- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22040cefd0 msgr2=0x7f2204146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f2209a1f640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22040cefd0 0x7f2204146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f2209a1f640 1 -- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f220415ada0 msgr2=0x7f2204146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f2209a1f640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f220415ada0 0x7f2204146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f2209a1f640 1 -- 172.21.15.72:0/269433183 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f8009400 con 0x7f22040712b0 2023-12-12T09:55:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f220a220640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f220415ada0 0x7f2204146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f2209a1f640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040712b0 0x7f2204140f00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f220000e8a0 tx=0x7f220000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f21f67fc640 1 -- 172.21.15.72:0/269433183 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22000094f0 con 0x7f22040712b0 2023-12-12T09:55:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f21f67fc640 1 -- 172.21.15.72:0/269433183 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220000c940 con 0x7f22040712b0 2023-12-12T09:55:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.401+0000 7f21f67fc640 1 -- 172.21.15.72:0/269433183 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220000ccf0 con 0x7f22040712b0 2023-12-12T09:55:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.402+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2204141440 con 0x7f22040712b0 2023-12-12T09:55:12.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.402+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22040cb3e0 con 0x7f22040712b0 2023-12-12T09:55:12.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.403+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22040673f0 con 0x7f22040712b0 2023-12-12T09:55:12.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.404+0000 7f21f67fc640 1 -- 172.21.15.72:0/269433183 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2200013470 con 0x7f22040712b0 2023-12-12T09:55:12.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.404+0000 7f21f67fc640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f21e806ae40 0x7f21e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.405+0000 7f21f67fc640 1 -- 172.21.15.72:0/269433183 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f22000895d0 con 0x7f22040712b0 2023-12-12T09:55:12.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.405+0000 7f220921e640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f21e806ae40 0x7f21e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.405+0000 7f220921e640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f21e806ae40 0x7f21e806d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f21f8036000 tx=0x7f21f8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.409+0000 7f21f67fc640 1 -- 172.21.15.72:0/269433183 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22000547a0 con 0x7f22040712b0 2023-12-12T09:55:12.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.601+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f22040ce7d0 con 0x7f21e806ae40 2023-12-12T09:55:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.602+0000 7f21f67fc640 1 -- 172.21.15.72:0/269433183 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f21f8031b00 con 0x7f21e806ae40 2023-12-12T09:55:12.604 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 3 to deep-scrub 2023-12-12T09:55:12.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f21e806ae40 msgr2=0x7f21e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f21e806ae40 0x7f21e806d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f21f8036000 tx=0x7f21f8008030 comp rx=0 tx=0).stop 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040712b0 msgr2=0x7f2204140f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040712b0 0x7f2204140f00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f220000e8a0 tx=0x7f220000ed60 comp rx=0 tx=0).stop 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 shutdown_connections 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f21e806ae40 0x7f21e806d300 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f220415ada0 0x7f2204146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22040cefd0 0x7f2204146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 --2- 172.21.15.72:0/269433183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22040712b0 0x7f2204140f00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 >> 172.21.15.72:0/269433183 conn(0x7f22040b1540 msgr2=0x7f2204158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:12.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 shutdown_connections 2023-12-12T09:55:12.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.605+0000 7f220aa21640 1 -- 172.21.15.72:0/269433183 wait complete. 2023-12-12T09:55:12.626 INFO:tasks.ceph:Scrubbing osd.4 2023-12-12T09:55:12.627 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-12T09:55:12.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.843+0000 7fd20a836640 1 Processor -- start 2023-12-12T09:55:12.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.844+0000 7fd20a836640 1 -- start start 2023-12-12T09:55:12.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.845+0000 7fd20a836640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd2041448a0 0x7fd20406c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.845+0000 7fd20a836640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20415ec10 0x7fd20415f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.845+0000 7fd20a836640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2040b3c50 con 0x7fd2040724a0 2023-12-12T09:55:12.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.845+0000 7fd20a836640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2040b3960 con 0x7fd2041448a0 2023-12-12T09:55:12.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.845+0000 7fd20a836640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20415f550 con 0x7fd20415ec10 2023-12-12T09:55:12.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.846+0000 7fd203fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd2041448a0 0x7fd20406c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.846+0000 7fd203fff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd2041448a0 0x7fd20406c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37028/0 (socket says 172.21.15.72:37028) 2023-12-12T09:55:12.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.846+0000 7fd203fff640 1 -- 172.21.15.72:0/2184437302 learned_addr learned my addr 172.21.15.72:0/2184437302 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:12.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.846+0000 7fd203fff640 1 -- 172.21.15.72:0/2184437302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20415ec10 msgr2=0x7fd20415f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:12.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.846+0000 7fd203fff640 1 --2- 172.21.15.72:0/2184437302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20415ec10 0x7fd20415f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.846+0000 7fd203fff640 1 -- 172.21.15.72:0/2184437302 >> v1:172.21.15.113:6789/0 conn(0x7fd2040724a0 legacy=0x7fd2040b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:12.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.846+0000 7fd203fff640 1 -- 172.21.15.72:0/2184437302 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204071140 con 0x7fd2041448a0 2023-12-12T09:55:12.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.847+0000 7fd203fff640 1 --2- 172.21.15.72:0/2184437302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 0x7fd20406c430 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009870 tx=0x7fd1f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b2dea22ba532f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd2037fe640 1 -- 172.21.15.72:0/2184437302 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4031640 con 0x7fd2041448a0 2023-12-12T09:55:12.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd2037fe640 1 -- 172.21.15.72:0/2184437302 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f40317a0 con 0x7fd2041448a0 2023-12-12T09:55:12.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd2037fe640 1 -- 172.21.15.72:0/2184437302 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4031b30 con 0x7fd2041448a0 2023-12-12T09:55:12.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd20a836640 1 -- 172.21.15.72:0/2184437302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 msgr2=0x7fd20406c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd20a836640 1 --2- 172.21.15.72:0/2184437302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 0x7fd20406c430 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009870 tx=0x7fd1f402b8f0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd20a836640 1 -- 172.21.15.72:0/2184437302 shutdown_connections 2023-12-12T09:55:12.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd20a836640 1 --2- 172.21.15.72:0/2184437302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20415ec10 0x7fd20415f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd20a836640 1 --2- 172.21.15.72:0/2184437302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 0x7fd20406c430 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd20a836640 1 -- 172.21.15.72:0/2184437302 >> 172.21.15.72:0/2184437302 conn(0x7fd2040b0a90 msgr2=0x7fd2040b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:12.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.848+0000 7fd20a836640 1 -- 172.21.15.72:0/2184437302 shutdown_connections 2023-12-12T09:55:12.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.849+0000 7fd20a836640 1 -- 172.21.15.72:0/2184437302 wait complete. 2023-12-12T09:55:12.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.850+0000 7fd20a836640 1 Processor -- start 2023-12-12T09:55:12.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.850+0000 7fd20a836640 1 -- start start 2023-12-12T09:55:12.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.851+0000 7fd20a836640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2040724a0 0x7fd2040ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.851+0000 7fd20a836640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 0x7fd20415d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.851+0000 7fd20a836640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20415ec10 0x7fd2040af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.851+0000 7fd20a836640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204140000 con 0x7fd2041448a0 2023-12-12T09:55:12.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.851+0000 7fd20a836640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20415e520 con 0x7fd2040724a0 2023-12-12T09:55:12.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.851+0000 7fd20a836640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20415e240 con 0x7fd20415ec10 2023-12-12T09:55:12.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 0x7fd20415d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 0x7fd20415d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37042/0 (socket says 172.21.15.72:37042) 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 -- 172.21.15.72:0/4294337552 learned_addr learned my addr 172.21.15.72:0/4294337552 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd208dd3640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20415ec10 0x7fd2040af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 -- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20415ec10 msgr2=0x7fd2040af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd203fff640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2040724a0 0x7fd2040ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20415ec10 0x7fd2040af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 -- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2040724a0 msgr2=0x7fd2040ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2040724a0 0x7fd2040ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 -- 172.21.15.72:0/4294337552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f40094d0 con 0x7fd2041448a0 2023-12-12T09:55:12.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd203fff640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2040724a0 0x7fd2040ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:12.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd1fbfff640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 0x7fd20415d8d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c400 tx=0x7fd1ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.852+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0035f0 con 0x7fd2041448a0 2023-12-12T09:55:12.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.853+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec003750 con 0x7fd2041448a0 2023-12-12T09:55:12.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.853+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2041f1e20 con 0x7fd2041448a0 2023-12-12T09:55:12.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.853+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec003b00 con 0x7fd2041448a0 2023-12-12T09:55:12.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.853+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2041f23e0 con 0x7fd2041448a0 2023-12-12T09:55:12.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.854+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2040b24a0 con 0x7fd2041448a0 2023-12-12T09:55:12.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.855+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1ec003c60 con 0x7fd2041448a0 2023-12-12T09:55:12.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.855+0000 7fd2017fa640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1e406ae40 0x7fd1e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.856+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd1ec088a10 con 0x7fd2041448a0 2023-12-12T09:55:12.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.856+0000 7fd203fff640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1e406ae40 0x7fd1e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.856+0000 7fd2017fa640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7fd1e4071140 0x7fd1e4073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:12.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.856+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 --> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1e4073c50 con 0x7fd1e4071140 2023-12-12T09:55:12.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.856+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1ec08d2a0 con 0x7fd2041448a0 2023-12-12T09:55:12.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.856+0000 7fd203fff640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1e406ae40 0x7fd1e406d300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4036000 tx=0x7fd1f403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.857+0000 7fd208dd3640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7fd1e4071140 0x7fd1e4073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:12.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.857+0000 7fd208dd3640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7fd1e4071140 0x7fd1e4073580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:12.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.859+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 <== osd.4 v2:172.21.15.113:6800/1025641669 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd1e4073c50 con 0x7fd1e4071140 2023-12-12T09:55:12.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.874+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 --> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd204071700 con 0x7fd1e4071140 2023-12-12T09:55:12.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd2017fa640 1 -- 172.21.15.72:0/4294337552 <== osd.4 v2:172.21.15.113:6800/1025641669 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd1fc007930 con 0x7fd1e4071140 2023-12-12T09:55:12.877 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:55:12.877 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-12T09:55:12.877 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7fd1e4071140 msgr2=0x7fd1e4073580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd20a836640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.113:6800/1025641669,v1:172.21.15.113:6801/1025641669] conn(0x7fd1e4071140 0x7fd1e4073580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1e406ae40 msgr2=0x7fd1e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd20a836640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fd1e406ae40 0x7fd1e406d300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4036000 tx=0x7fd1f403c040 comp rx=0 tx=0).stop 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 msgr2=0x7fd20415d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd20a836640 1 --2- 172.21.15.72:0/4294337552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2041448a0 0x7fd20415d8d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c400 tx=0x7fd1ec00c8c0 comp rx=0 tx=0).stop 2023-12-12T09:55:12.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd208dd3640 1 -- 172.21.15.72:0/4294337552 reap_dead start 2023-12-12T09:55:12.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 shutdown_connections 2023-12-12T09:55:12.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.875+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 >> 172.21.15.72:0/4294337552 conn(0x7fd2040b0a90 msgr2=0x7fd2040ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:12.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.876+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 shutdown_connections 2023-12-12T09:55:12.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:12.876+0000 7fd20a836640 1 -- 172.21.15.72:0/4294337552 wait complete. 2023-12-12T09:55:12.894 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-12T09:55:13.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.111+0000 7ffa3ba48640 1 Processor -- start 2023-12-12T09:55:13.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.111+0000 7ffa3ba48640 1 -- start start 2023-12-12T09:55:13.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.112+0000 7ffa3ba48640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa341607f0 0x7ffa340ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.112+0000 7ffa3ba48640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa340cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.112+0000 7ffa3ba48640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa340d0370 con 0x7ffa340cff00 2023-12-12T09:55:13.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.112+0000 7ffa3ba48640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa340d1be0 con 0x7ffa341607f0 2023-12-12T09:55:13.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.112+0000 7ffa3ba48640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340cf110 con 0x7ffa340cecb0 2023-12-12T09:55:13.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.113+0000 7ffa38fe3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa340cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.113+0000 7ffa397e4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa341607f0 0x7ffa340ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.113+0000 7ffa38fe3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa340cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:36666/0 (socket says 172.21.15.72:36666) 2023-12-12T09:55:13.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.113+0000 7ffa38fe3640 1 -- 172.21.15.72:0/1362084795 learned_addr learned my addr 172.21.15.72:0/1362084795 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:13.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.114+0000 7ffa397e4640 1 -- 172.21.15.72:0/1362084795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 msgr2=0x7ffa340cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:13.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.114+0000 7ffa397e4640 1 --2- 172.21.15.72:0/1362084795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa340cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.114+0000 7ffa397e4640 1 -- 172.21.15.72:0/1362084795 >> v1:172.21.15.113:6789/0 conn(0x7ffa340cff00 legacy=0x7ffa34160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.114+0000 7ffa397e4640 1 -- 172.21.15.72:0/1362084795 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa340b6fe0 con 0x7ffa341607f0 2023-12-12T09:55:13.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.115+0000 7ffa397e4640 1 --2- 172.21.15.72:0/1362084795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa341607f0 0x7ffa340ce770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009dc0 tx=0x7ffa2802be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9643431ddaf77cf server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.116+0000 7ffa33fff640 1 -- 172.21.15.72:0/1362084795 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2802cbb0 con 0x7ffa341607f0 2023-12-12T09:55:13.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.116+0000 7ffa33fff640 1 -- 172.21.15.72:0/1362084795 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2802cd10 con 0x7ffa341607f0 2023-12-12T09:55:13.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.116+0000 7ffa33fff640 1 -- 172.21.15.72:0/1362084795 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28034660 con 0x7ffa341607f0 2023-12-12T09:55:13.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.117+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1362084795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa341607f0 msgr2=0x7ffa340ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.117+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1362084795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa341607f0 0x7ffa340ce770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009dc0 tx=0x7ffa2802be00 comp rx=0 tx=0).stop 2023-12-12T09:55:13.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.117+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1362084795 shutdown_connections 2023-12-12T09:55:13.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.117+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1362084795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa340cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.117+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1362084795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa341607f0 0x7ffa340ce770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.117+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1362084795 >> 172.21.15.72:0/1362084795 conn(0x7ffa3409c450 msgr2=0x7ffa340d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:13.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.117+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1362084795 shutdown_connections 2023-12-12T09:55:13.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.117+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1362084795 wait complete. 2023-12-12T09:55:13.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.119+0000 7ffa3ba48640 1 Processor -- start 2023-12-12T09:55:13.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.119+0000 7ffa3ba48640 1 -- start start 2023-12-12T09:55:13.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa3ba48640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa3414bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa3ba48640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa340cff00 0x7ffa34148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa3ba48640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa341607f0 0x7ffa341e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa3ba48640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340c8be0 con 0x7ffa341607f0 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa3ba48640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3414c0c0 con 0x7ffa340cff00 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa3ba48640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa34148dd0 con 0x7ffa340cecb0 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa39fe5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa341607f0 0x7ffa341e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa39fe5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa341607f0 0x7ffa341e5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37044/0 (socket says 172.21.15.72:37044) 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa39fe5640 1 -- 172.21.15.72:0/1872004749 learned_addr learned my addr 172.21.15.72:0/1872004749 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.120+0000 7ffa397e4640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa3414bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa38fe3640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa340cff00 0x7ffa34148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa39fe5640 1 -- 172.21.15.72:0/1872004749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 msgr2=0x7ffa3414bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa39fe5640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa3414bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa39fe5640 1 -- 172.21.15.72:0/1872004749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa340cff00 msgr2=0x7ffa34148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa39fe5640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa340cff00 0x7ffa34148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa39fe5640 1 -- 172.21.15.72:0/1872004749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28009a70 con 0x7ffa341607f0 2023-12-12T09:55:13.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa39fe5640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa341607f0 0x7ffa341e5af0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00c3f0 tx=0x7ffa2c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa31ffb640 1 -- 172.21.15.72:0/1872004749 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c0052d0 con 0x7ffa341607f0 2023-12-12T09:55:13.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa31ffb640 1 -- 172.21.15.72:0/1872004749 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c005430 con 0x7ffa341607f0 2023-12-12T09:55:13.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa340c7fd0 con 0x7ffa341607f0 2023-12-12T09:55:13.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa31ffb640 1 -- 172.21.15.72:0/1872004749 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c003850 con 0x7ffa341607f0 2023-12-12T09:55:13.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.121+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa340c8590 con 0x7ffa341607f0 2023-12-12T09:55:13.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.122+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa340c8d90 con 0x7ffa341607f0 2023-12-12T09:55:13.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.123+0000 7ffa31ffb640 1 -- 172.21.15.72:0/1872004749 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa2c0039b0 con 0x7ffa341607f0 2023-12-12T09:55:13.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.124+0000 7ffa31ffb640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa1806ae40 0x7ffa1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.124+0000 7ffa31ffb640 1 -- 172.21.15.72:0/1872004749 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ffa2c088a10 con 0x7ffa341607f0 2023-12-12T09:55:13.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.125+0000 7ffa397e4640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa1806ae40 0x7ffa1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.125+0000 7ffa397e4640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa1806ae40 0x7ffa1806d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa28036000 tx=0x7ffa28030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.128+0000 7ffa31ffb640 1 -- 172.21.15.72:0/1872004749 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa340c8d90 con 0x7ffa341607f0 2023-12-12T09:55:13.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.318+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa340d1520 con 0x7ffa1806ae40 2023-12-12T09:55:13.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.319+0000 7ffa31ffb640 1 -- 172.21.15.72:0/1872004749 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffa2802c9b0 con 0x7ffa1806ae40 2023-12-12T09:55:13.321 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 4 to deep-scrub 2023-12-12T09:55:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.321+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa1806ae40 msgr2=0x7ffa1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.321+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa1806ae40 0x7ffa1806d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa28036000 tx=0x7ffa28030000 comp rx=0 tx=0).stop 2023-12-12T09:55:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.321+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa341607f0 msgr2=0x7ffa341e5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.321+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa341607f0 0x7ffa341e5af0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00c3f0 tx=0x7ffa2c00c8b0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.322+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 shutdown_connections 2023-12-12T09:55:13.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.322+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ffa1806ae40 0x7ffa1806d300 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.322+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa341607f0 0x7ffa341e5af0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.322+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa340cff00 0x7ffa34148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.322+0000 7ffa3ba48640 1 --2- 172.21.15.72:0/1872004749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cecb0 0x7ffa3414bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.322+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 >> 172.21.15.72:0/1872004749 conn(0x7ffa3409c450 msgr2=0x7ffa340b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:13.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.322+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 shutdown_connections 2023-12-12T09:55:13.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.322+0000 7ffa3ba48640 1 -- 172.21.15.72:0/1872004749 wait complete. 2023-12-12T09:55:13.342 INFO:tasks.ceph:Scrubbing osd.5 2023-12-12T09:55:13.343 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-12T09:55:13.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.558+0000 7fb7a631d640 1 Processor -- start 2023-12-12T09:55:13.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.558+0000 7fb7a631d640 1 -- start start 2023-12-12T09:55:13.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb7a631d640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb7a0159ed0 0x7fb7a006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb7a631d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7a00cd990 0x7fb7a015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb7a631d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a0070fc0 con 0x7fb7a015db60 2023-12-12T09:55:13.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb7a631d640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a015e530 con 0x7fb7a0159ed0 2023-12-12T09:55:13.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb7a631d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00712a0 con 0x7fb7a00cd990 2023-12-12T09:55:13.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb79ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb7a0159ed0 0x7fb7a006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb79ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb7a0159ed0 0x7fb7a006ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37046/0 (socket says 172.21.15.72:37046) 2023-12-12T09:55:13.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb79ffff640 1 -- 172.21.15.72:0/430569845 learned_addr learned my addr 172.21.15.72:0/430569845 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:13.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.560+0000 7fb7a48ba640 1 --2- 172.21.15.72:0/430569845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7a00cd990 0x7fb7a015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.561+0000 7fb79ffff640 1 -- 172.21.15.72:0/430569845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7a00cd990 msgr2=0x7fb7a015dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:13.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.561+0000 7fb79ffff640 1 --2- 172.21.15.72:0/430569845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7a00cd990 0x7fb7a015dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.561+0000 7fb79ffff640 1 -- 172.21.15.72:0/430569845 >> v1:172.21.15.113:6789/0 conn(0x7fb7a015db60 legacy=0x7fb7a015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:13.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.561+0000 7fb79ffff640 1 -- 172.21.15.72:0/430569845 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a0071580 con 0x7fb7a0159ed0 2023-12-12T09:55:13.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.562+0000 7fb79ffff640 1 --2- 172.21.15.72:0/430569845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 0x7fb7a006ddd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb79000bb10 tx=0x7fb79002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5041597ba3af691e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.562+0000 7fb79effd640 1 -- 172.21.15.72:0/430569845 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79002ee70 con 0x7fb7a0159ed0 2023-12-12T09:55:13.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.562+0000 7fb79effd640 1 -- 172.21.15.72:0/430569845 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790036440 con 0x7fb7a0159ed0 2023-12-12T09:55:13.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.562+0000 7fb79effd640 1 -- 172.21.15.72:0/430569845 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900367d0 con 0x7fb7a0159ed0 2023-12-12T09:55:13.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.563+0000 7fb7a631d640 1 -- 172.21.15.72:0/430569845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 msgr2=0x7fb7a006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.563+0000 7fb7a631d640 1 --2- 172.21.15.72:0/430569845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 0x7fb7a006ddd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb79000bb10 tx=0x7fb79002e0c0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.563+0000 7fb7a631d640 1 -- 172.21.15.72:0/430569845 shutdown_connections 2023-12-12T09:55:13.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.563+0000 7fb7a631d640 1 --2- 172.21.15.72:0/430569845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7a00cd990 0x7fb7a015dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.563+0000 7fb7a631d640 1 --2- 172.21.15.72:0/430569845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 0x7fb7a006ddd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.563+0000 7fb7a631d640 1 -- 172.21.15.72:0/430569845 >> 172.21.15.72:0/430569845 conn(0x7fb7a00b0a90 msgr2=0x7fb7a00b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:13.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.563+0000 7fb7a631d640 1 -- 172.21.15.72:0/430569845 shutdown_connections 2023-12-12T09:55:13.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.563+0000 7fb7a631d640 1 -- 172.21.15.72:0/430569845 wait complete. 2023-12-12T09:55:13.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.565+0000 7fb7a631d640 1 Processor -- start 2023-12-12T09:55:13.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.565+0000 7fb7a631d640 1 -- start start 2023-12-12T09:55:13.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb7a631d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7a00cd990 0x7fb7a00c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb7a631d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 0x7fb7a00caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb7a631d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a015db60 0x7fb7a0141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb7a631d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a013fed0 con 0x7fb7a0159ed0 2023-12-12T09:55:13.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb7a631d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00c4240 con 0x7fb7a015db60 2023-12-12T09:55:13.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb7a631d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00cb470 con 0x7fb7a00cd990 2023-12-12T09:55:13.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb79f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 0x7fb7a00caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb79f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 0x7fb7a00caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37058/0 (socket says 172.21.15.72:37058) 2023-12-12T09:55:13.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.566+0000 7fb79f7fe640 1 -- 172.21.15.72:0/1461714608 learned_addr learned my addr 172.21.15.72:0/1461714608 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:13.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79f7fe640 1 -- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7a00cd990 msgr2=0x7fb7a00c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79f7fe640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7a00cd990 0x7fb7a00c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79f7fe640 1 -- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a015db60 msgr2=0x7fb7a0141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79f7fe640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7a015db60 0x7fb7a0141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79f7fe640 1 -- 172.21.15.72:0/1461714608 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79000b770 con 0x7fb7a0159ed0 2023-12-12T09:55:13.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79f7fe640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 0x7fb7a00caf30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b420 tx=0x7fb79400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794011830 con 0x7fb7a0159ed0 2023-12-12T09:55:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794011990 con 0x7fb7a0159ed0 2023-12-12T09:55:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a0159510 con 0x7fb7a0159ed0 2023-12-12T09:55:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794011d40 con 0x7fb7a0159ed0 2023-12-12T09:55:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.567+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a013faf0 con 0x7fb7a0159ed0 2023-12-12T09:55:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.568+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7a013f690 con 0x7fb7a0159ed0 2023-12-12T09:55:13.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.569+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb79401b460 con 0x7fb7a0159ed0 2023-12-12T09:55:13.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.570+0000 7fb79cff9640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb78c06ae40 0x7fb78c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.570+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb794088ef0 con 0x7fb7a0159ed0 2023-12-12T09:55:13.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.571+0000 7fb79ffff640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb78c06ae40 0x7fb78c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.571+0000 7fb79cff9640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fb78c071140 0x7fb78c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.571+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 --> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb78c073c50 con 0x7fb78c071140 2023-12-12T09:55:13.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.571+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79408b5b0 con 0x7fb7a0159ed0 2023-12-12T09:55:13.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.571+0000 7fb79ffff640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb78c06ae40 0x7fb78c06d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb790038000 tx=0x7fb790030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.571+0000 7fb7a48ba640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fb78c071140 0x7fb78c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.572+0000 7fb7a48ba640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fb78c071140 0x7fb78c073580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.575+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 <== osd.5 v2:172.21.15.113:6808/1716073726 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb78c073c50 con 0x7fb78c071140 2023-12-12T09:55:13.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.589+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 --> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb7a015e8b0 con 0x7fb78c071140 2023-12-12T09:55:13.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb79cff9640 1 -- 172.21.15.72:0/1461714608 <== osd.5 v2:172.21.15.113:6808/1716073726 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb798009ad0 con 0x7fb78c071140 2023-12-12T09:55:13.592 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:55:13.592 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-12T09:55:13.592 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:55:13.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fb78c071140 msgr2=0x7fb78c073580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a631d640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.113:6808/1716073726,v1:172.21.15.113:6809/1716073726] conn(0x7fb78c071140 0x7fb78c073580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb78c06ae40 msgr2=0x7fb78c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a631d640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7fb78c06ae40 0x7fb78c06d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb790038000 tx=0x7fb790030000 comp rx=0 tx=0).stop 2023-12-12T09:55:13.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 msgr2=0x7fb7a00caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a631d640 1 --2- 172.21.15.72:0/1461714608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7a0159ed0 0x7fb7a00caf30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b420 tx=0x7fb79400b8e0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a48ba640 1 -- 172.21.15.72:0/1461714608 reap_dead start 2023-12-12T09:55:13.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 shutdown_connections 2023-12-12T09:55:13.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.590+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 >> 172.21.15.72:0/1461714608 conn(0x7fb7a00b0a90 msgr2=0x7fb7a00b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:13.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.591+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 shutdown_connections 2023-12-12T09:55:13.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.591+0000 7fb7a631d640 1 -- 172.21.15.72:0/1461714608 wait complete. 2023-12-12T09:55:13.610 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-12T09:55:13.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.818+0000 7efe81890640 1 Processor -- start 2023-12-12T09:55:13.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.818+0000 7efe81890640 1 -- start start 2023-12-12T09:55:13.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.820+0000 7efe81890640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe7c15e2b0 0x7efe7c164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.820+0000 7efe81890640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7c06ca40 con 0x7efe7c157a60 2023-12-12T09:55:13.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.820+0000 7efe81890640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7c071cd0 con 0x7efe7c06c410 2023-12-12T09:55:13.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.820+0000 7efe81890640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7c070f10 con 0x7efe7c15e2b0 2023-12-12T09:55:13.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.820+0000 7efe7affd640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7efe7c157a60 0x7efe7c06de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43124/0 (socket says 172.21.15.72:43124) 2023-12-12T09:55:13.822 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.820+0000 7efe7affd640 1 -- 172.21.15.72:0/3136822165 learned_addr learned my addr 172.21.15.72:0/3136822165 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:13.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7b7fe640 1 --2- 172.21.15.72:0/3136822165 >> v2:172.21.15.113:3300/0 conn(0x7efe7c15e2b0 0x7efe7c164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933469464 0 0) 0x7efe7c06ca40 con 0x7efe7c157a60 2023-12-12T09:55:13.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe68003520 con 0x7efe7c157a60 2023-12-12T09:55:13.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3841351162 0 0) 0x7efe68003520 con 0x7efe7c157a60 2023-12-12T09:55:13.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 >> v2:172.21.15.113:3300/0 conn(0x7efe7c15e2b0 msgr2=0x7efe7c164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:13.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 --2- 172.21.15.72:0/3136822165 >> v2:172.21.15.113:3300/0 conn(0x7efe7c15e2b0 0x7efe7c164d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 >> v1:172.21.15.113:6789/0 conn(0x7efe7c06c410 legacy=0x7efe7c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:13.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c0711f0 con 0x7efe7c157a60 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7efe6c001670 con 0x7efe7c157a60 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 >> v1:172.21.15.72:6789/0 conn(0x7efe7c157a60 legacy=0x7efe7c06de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe68003970 0x7efe68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe680047c0 0x7efe68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe68005180 0x7efe6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c070f10 con 0x7efe68003970 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6800c120 con 0x7efe680047c0 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.821+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6800c2e0 con 0x7efe68005180 2023-12-12T09:55:13.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe7affd640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe68003970 0x7efe68003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe73fff640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe680047c0 0x7efe68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe7affd640 1 -- 172.21.15.72:0/3136822165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe68005180 msgr2=0x7efe6800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:13.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe7b7fe640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe68005180 0x7efe6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe7affd640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe68005180 0x7efe6800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe7affd640 1 -- 172.21.15.72:0/3136822165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe680047c0 msgr2=0x7efe68004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:13.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe7affd640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe680047c0 0x7efe68004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe7affd640 1 -- 172.21.15.72:0/3136822165 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68003520 con 0x7efe68003970 2023-12-12T09:55:13.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.822+0000 7efe7b7fe640 1 -- 172.21.15.72:0/3136822165 reap_dead start 2023-12-12T09:55:13.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.823+0000 7efe7affd640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe68003970 0x7efe68003d90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00c240 tx=0x7efe680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807bee428ec1360f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c005ad0 con 0x7efe68003970 2023-12-12T09:55:13.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c005c30 con 0x7efe68003970 2023-12-12T09:55:13.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe7a7fc640 1 -- 172.21.15.72:0/3136822165 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c02ca50 con 0x7efe68003970 2023-12-12T09:55:13.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe81890640 1 -- 172.21.15.72:0/3136822165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe68003970 msgr2=0x7efe68003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe81890640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe68003970 0x7efe68003d90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00c240 tx=0x7efe680057f0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe81890640 1 -- 172.21.15.72:0/3136822165 shutdown_connections 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe81890640 1 --2- 172.21.15.72:0/3136822165 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe68003970 0x7efe68003d90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe81890640 1 -- 172.21.15.72:0/3136822165 >> 172.21.15.72:0/3136822165 conn(0x7efe7c0b0a90 msgr2=0x7efe7c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe81890640 1 -- 172.21.15.72:0/3136822165 shutdown_connections 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.824+0000 7efe81890640 1 -- 172.21.15.72:0/3136822165 wait complete. 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.826+0000 7efe81890640 1 Processor -- start 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.826+0000 7efe81890640 1 -- start start 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.827+0000 7efe81890640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe68003970 0x7efe7c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.827+0000 7efe81890640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe7c15d8b0 0x7efe7c14f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.827+0000 7efe81890640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe7c1476a0 0x7efe7c159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.827+0000 7efe81890640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c15dd30 con 0x7efe7c15d8b0 2023-12-12T09:55:13.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.827+0000 7efe81890640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c14fc10 con 0x7efe68003970 2023-12-12T09:55:13.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.827+0000 7efe81890640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0b45c0 con 0x7efe7c1476a0 2023-12-12T09:55:13.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.827+0000 7efe73fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe7c15d8b0 0x7efe7c14f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.827+0000 7efe73fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe7c15d8b0 0x7efe7c14f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37090/0 (socket says 172.21.15.72:37090) 2023-12-12T09:55:13.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe73fff640 1 -- 172.21.15.72:0/781919037 learned_addr learned my addr 172.21.15.72:0/781919037 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe73fff640 1 -- 172.21.15.72:0/781919037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe7c1476a0 msgr2=0x7efe7c159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe73fff640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe7c1476a0 0x7efe7c159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe73fff640 1 -- 172.21.15.72:0/781919037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe68003970 msgr2=0x7efe7c15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe73fff640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe68003970 0x7efe7c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe73fff640 1 -- 172.21.15.72:0/781919037 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c0b48a0 con 0x7efe7c15d8b0 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe73fff640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe7c15d8b0 0x7efe7c14f6d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e410 tx=0x7efe6400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe78ff9640 1 -- 172.21.15.72:0/781919037 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe640052e0 con 0x7efe7c15d8b0 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe78ff9640 1 -- 172.21.15.72:0/781919037 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64005440 con 0x7efe7c15d8b0 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe78ff9640 1 -- 172.21.15.72:0/781919037 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe640037f0 con 0x7efe7c15d8b0 2023-12-12T09:55:13.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.828+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c0b4b80 con 0x7efe7c15d8b0 2023-12-12T09:55:13.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.829+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c15c0c0 con 0x7efe7c15d8b0 2023-12-12T09:55:13.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.830+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c0745d0 con 0x7efe7c15d8b0 2023-12-12T09:55:13.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.831+0000 7efe78ff9640 1 -- 172.21.15.72:0/781919037 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe64003950 con 0x7efe7c15d8b0 2023-12-12T09:55:13.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.831+0000 7efe78ff9640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe5c06ae40 0x7efe5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:13.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.832+0000 7efe78ff9640 1 -- 172.21.15.72:0/781919037 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7efe64088a90 con 0x7efe7c15d8b0 2023-12-12T09:55:13.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.832+0000 7efe7affd640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe5c06ae40 0x7efe5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:13.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.832+0000 7efe7affd640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe5c06ae40 0x7efe5c06d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efe6c034000 tx=0x7efe6c039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:13.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:13.837+0000 7efe78ff9640 1 -- 172.21.15.72:0/781919037 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe64053e40 con 0x7efe7c15d8b0 2023-12-12T09:55:14.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.027+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7efe7c0717b0 con 0x7efe5c06ae40 2023-12-12T09:55:14.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.028+0000 7efe78ff9640 1 -- 172.21.15.72:0/781919037 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efe6c001430 con 0x7efe5c06ae40 2023-12-12T09:55:14.030 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 5 to deep-scrub 2023-12-12T09:55:14.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.030+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe5c06ae40 msgr2=0x7efe5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.030+0000 7efe81890640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe5c06ae40 0x7efe5c06d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efe6c034000 tx=0x7efe6c039040 comp rx=0 tx=0).stop 2023-12-12T09:55:14.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.030+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe7c15d8b0 msgr2=0x7efe7c14f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.030+0000 7efe81890640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe7c15d8b0 0x7efe7c14f6d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e410 tx=0x7efe6400e8d0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.031+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 shutdown_connections 2023-12-12T09:55:14.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.031+0000 7efe81890640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7efe5c06ae40 0x7efe5c06d300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.031+0000 7efe81890640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe7c1476a0 0x7efe7c159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.031+0000 7efe81890640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe7c15d8b0 0x7efe7c14f6d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.031+0000 7efe81890640 1 --2- 172.21.15.72:0/781919037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe68003970 0x7efe7c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.031+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 >> 172.21.15.72:0/781919037 conn(0x7efe7c0b0a90 msgr2=0x7efe7c070180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:14.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.031+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 shutdown_connections 2023-12-12T09:55:14.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.031+0000 7efe81890640 1 -- 172.21.15.72:0/781919037 wait complete. 2023-12-12T09:55:14.051 INFO:tasks.ceph:Scrubbing osd.6 2023-12-12T09:55:14.051 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-12T09:55:14.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.266+0000 7f1295256640 1 Processor -- start 2023-12-12T09:55:14.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.266+0000 7f1295256640 1 -- start start 2023-12-12T09:55:14.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.268+0000 7f1295256640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f12900cdab0 0x7f1290070200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.268+0000 7f1295256640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdf10 0x7f1290070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.268+0000 7f1295256640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129006c430 con 0x7f129014b090 2023-12-12T09:55:14.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.268+0000 7f1295256640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12900ccf40 con 0x7f12900cdab0 2023-12-12T09:55:14.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.268+0000 7f1295256640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12900b24a0 con 0x7f12900cdf10 2023-12-12T09:55:14.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.268+0000 7f128ed9d640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f129014b090 0x7f12900b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43132/0 (socket says 172.21.15.72:43132) 2023-12-12T09:55:14.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.268+0000 7f128e59c640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f12900cdab0 0x7f1290070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.268+0000 7f128ed9d640 1 -- 172.21.15.72:0/83381768 learned_addr learned my addr 172.21.15.72:0/83381768 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:14.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.269+0000 7f128f59e640 1 --2- 172.21.15.72:0/83381768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdf10 0x7f1290070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.269+0000 7f128dd9b640 1 -- 172.21.15.72:0/83381768 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759953862 0 0) 0x7f129006c430 con 0x7f129014b090 2023-12-12T09:55:14.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.269+0000 7f128dd9b640 1 -- 172.21.15.72:0/83381768 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f127c003520 con 0x7f129014b090 2023-12-12T09:55:14.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.269+0000 7f128e59c640 1 -- 172.21.15.72:0/83381768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdf10 msgr2=0x7f1290070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:14.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.269+0000 7f128e59c640 1 --2- 172.21.15.72:0/83381768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdf10 0x7f1290070740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.269+0000 7f128e59c640 1 -- 172.21.15.72:0/83381768 >> v1:172.21.15.72:6789/0 conn(0x7f129014b090 legacy=0x7f12900b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.269+0000 7f128e59c640 1 -- 172.21.15.72:0/83381768 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129006c710 con 0x7f12900cdab0 2023-12-12T09:55:14.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.270+0000 7f128e59c640 1 --2- 172.21.15.72:0/83381768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdab0 0x7f1290070200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f12780097a0 tx=0x7f127802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95720d7c8fe10053 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f128dd9b640 1 -- 172.21.15.72:0/83381768 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278031830 con 0x7f12900cdab0 2023-12-12T09:55:14.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f128dd9b640 1 -- 172.21.15.72:0/83381768 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1278031990 con 0x7f12900cdab0 2023-12-12T09:55:14.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f128dd9b640 1 -- 172.21.15.72:0/83381768 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278031d20 con 0x7f12900cdab0 2023-12-12T09:55:14.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f1295256640 1 -- 172.21.15.72:0/83381768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdab0 msgr2=0x7f1290070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f1295256640 1 --2- 172.21.15.72:0/83381768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdab0 0x7f1290070200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f12780097a0 tx=0x7f127802b8d0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f1295256640 1 -- 172.21.15.72:0/83381768 shutdown_connections 2023-12-12T09:55:14.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f1295256640 1 --2- 172.21.15.72:0/83381768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdf10 0x7f1290070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f1295256640 1 --2- 172.21.15.72:0/83381768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdab0 0x7f1290070200 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.271+0000 7f1295256640 1 -- 172.21.15.72:0/83381768 >> 172.21.15.72:0/83381768 conn(0x7f12900b0a90 msgr2=0x7f12900b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:14.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.272+0000 7f1295256640 1 -- 172.21.15.72:0/83381768 shutdown_connections 2023-12-12T09:55:14.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.272+0000 7f1295256640 1 -- 172.21.15.72:0/83381768 wait complete. 2023-12-12T09:55:14.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.273+0000 7f1295256640 1 Processor -- start 2023-12-12T09:55:14.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.273+0000 7f1295256640 1 -- start start 2023-12-12T09:55:14.277 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.274+0000 7f1295256640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdab0 0x7f1290160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.274+0000 7f1295256640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdf10 0x7f12901f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.274+0000 7f1295256640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f129014b090 0x7f1290163260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.274+0000 7f1295256640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290142750 con 0x7f12900cdf10 2023-12-12T09:55:14.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.274+0000 7f1295256640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290142a30 con 0x7f129014b090 2023-12-12T09:55:14.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.274+0000 7f1295256640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1290142d10 con 0x7f12900cdab0 2023-12-12T09:55:14.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdf10 0x7f12901f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdf10 0x7f12901f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37106/0 (socket says 172.21.15.72:37106) 2023-12-12T09:55:14.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 -- 172.21.15.72:0/4121506867 learned_addr learned my addr 172.21.15.72:0/4121506867 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128ed9d640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdab0 0x7f1290160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 -- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdab0 msgr2=0x7f1290160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdab0 0x7f1290160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 -- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f129014b090 msgr2=0x7f1290163260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f129014b090 0x7f1290163260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 -- 172.21.15.72:0/4121506867 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278009400 con 0x7f12900cdf10 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128ed9d640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12900cdab0 0x7f1290160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.275+0000 7f128e59c640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdf10 0x7f12901f78c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1278034040 tx=0x7f12780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.276+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127803a8f0 con 0x7f12900cdf10 2023-12-12T09:55:14.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.276+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127803aa70 con 0x7f12900cdf10 2023-12-12T09:55:14.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.276+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1290142ff0 con 0x7f12900cdf10 2023-12-12T09:55:14.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.276+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127803ae20 con 0x7f12900cdf10 2023-12-12T09:55:14.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.276+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12901f73e0 con 0x7f12900cdf10 2023-12-12T09:55:14.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.277+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f129006cfb0 con 0x7f12900cdf10 2023-12-12T09:55:14.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.278+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f127804d440 con 0x7f12900cdf10 2023-12-12T09:55:14.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.279+0000 7f12877fe640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f127006ae70 0x7f127006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.279+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f12780abbf0 con 0x7f12900cdf10 2023-12-12T09:55:14.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.279+0000 7f128ed9d640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f127006ae70 0x7f127006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.279+0000 7f12877fe640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7f1270071170 0x7f12700735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.279+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 --> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1270073c80 con 0x7f1270071170 2023-12-12T09:55:14.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.279+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12780abf60 con 0x7f12900cdf10 2023-12-12T09:55:14.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.280+0000 7f128ed9d640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f127006ae70 0x7f127006d330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1280001560 tx=0x7f1280009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.280+0000 7f128f59e640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7f1270071170 0x7f12700735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.281+0000 7f128f59e640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7f1270071170 0x7f12700735b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.283+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 <== osd.6 v2:172.21.15.113:6816/3330758472 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1270073c80 con 0x7f1270071170 2023-12-12T09:55:14.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.297+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 --> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f129006ccd0 con 0x7f1270071170 2023-12-12T09:55:14.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.298+0000 7f12877fe640 1 -- 172.21.15.72:0/4121506867 <== osd.6 v2:172.21.15.113:6816/3330758472 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f12880099d0 con 0x7f1270071170 2023-12-12T09:55:14.300 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:55:14.300 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-12T09:55:14.301 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:55:14.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.298+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7f1270071170 msgr2=0x7f12700735b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.298+0000 7f1295256640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.113:6816/3330758472,v1:172.21.15.113:6817/3330758472] conn(0x7f1270071170 0x7f12700735b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.298+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f127006ae70 msgr2=0x7f127006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.298+0000 7f1295256640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f127006ae70 0x7f127006d330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1280001560 tx=0x7f1280009040 comp rx=0 tx=0).stop 2023-12-12T09:55:14.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.299+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdf10 msgr2=0x7f12901f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.299+0000 7f1295256640 1 --2- 172.21.15.72:0/4121506867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12900cdf10 0x7f12901f78c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1278034040 tx=0x7f12780046c0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.299+0000 7f128f59e640 1 -- 172.21.15.72:0/4121506867 reap_dead start 2023-12-12T09:55:14.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.299+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 shutdown_connections 2023-12-12T09:55:14.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.299+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 >> 172.21.15.72:0/4121506867 conn(0x7f12900b0a90 msgr2=0x7f12900b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:14.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.299+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 shutdown_connections 2023-12-12T09:55:14.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.299+0000 7f1295256640 1 -- 172.21.15.72:0/4121506867 wait complete. 2023-12-12T09:55:14.316 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-12T09:55:14.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.528+0000 7f3483741640 1 Processor -- start 2023-12-12T09:55:14.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.528+0000 7f3483741640 1 -- start start 2023-12-12T09:55:14.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.530+0000 7f3483741640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f347c157a60 0x7f347c06c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.530+0000 7f3483741640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f347c15e2b0 0x7f347c0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.530+0000 7f3483741640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c06ca40 con 0x7f347c06de20 2023-12-12T09:55:14.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.530+0000 7f3483741640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c071cd0 con 0x7f347c15e2b0 2023-12-12T09:55:14.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.530+0000 7f3483741640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c070f10 con 0x7f347c157a60 2023-12-12T09:55:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f3480cdc640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f347c06de20 0x7f347c15dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43142/0 (socket says 172.21.15.72:43142) 2023-12-12T09:55:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f3480cdc640 1 -- 172.21.15.72:0/2130238429 learned_addr learned my addr 172.21.15.72:0/2130238429 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f3481cde640 1 --2- 172.21.15.72:0/2130238429 >> v2:172.21.15.72:3300/0 conn(0x7f347c15e2b0 0x7f347c0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f34814dd640 1 --2- 172.21.15.72:0/2130238429 >> v2:172.21.15.113:3300/0 conn(0x7f347c157a60 0x7f347c06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f347bfff640 1 -- 172.21.15.72:0/2130238429 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341781942 0 0) 0x7f347c06ca40 con 0x7f347c06de20 2023-12-12T09:55:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f347bfff640 1 -- 172.21.15.72:0/2130238429 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c003520 con 0x7f347c06de20 2023-12-12T09:55:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f3481cde640 1 -- 172.21.15.72:0/2130238429 >> v2:172.21.15.113:3300/0 conn(0x7f347c157a60 msgr2=0x7f347c06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f3481cde640 1 --2- 172.21.15.72:0/2130238429 >> v2:172.21.15.113:3300/0 conn(0x7f347c157a60 0x7f347c06c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f3481cde640 1 -- 172.21.15.72:0/2130238429 >> v1:172.21.15.72:6789/0 conn(0x7f347c06de20 legacy=0x7f347c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.531+0000 7f3481cde640 1 -- 172.21.15.72:0/2130238429 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0711f0 con 0x7f347c15e2b0 2023-12-12T09:55:14.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.532+0000 7f3481cde640 1 --2- 172.21.15.72:0/2130238429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cc8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f347c0b1710 tx=0x7f347402dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5075fb67a3827f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f347bfff640 1 -- 172.21.15.72:0/2130238429 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347402eb20 con 0x7f347c15e2b0 2023-12-12T09:55:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f347bfff640 1 -- 172.21.15.72:0/2130238429 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347402ec80 con 0x7f347c15e2b0 2023-12-12T09:55:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f347bfff640 1 -- 172.21.15.72:0/2130238429 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474036440 con 0x7f347c15e2b0 2023-12-12T09:55:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f3483741640 1 -- 172.21.15.72:0/2130238429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 msgr2=0x7f347c0cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f3483741640 1 --2- 172.21.15.72:0/2130238429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cc8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f347c0b1710 tx=0x7f347402dd70 comp rx=0 tx=0).stop 2023-12-12T09:55:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f3483741640 1 -- 172.21.15.72:0/2130238429 shutdown_connections 2023-12-12T09:55:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f3483741640 1 --2- 172.21.15.72:0/2130238429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cc8f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f3483741640 1 --2- 172.21.15.72:0/2130238429 >> v2:172.21.15.113:3300/0 conn(0x7f347c157a60 0x7f347c06c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.533+0000 7f3483741640 1 -- 172.21.15.72:0/2130238429 >> 172.21.15.72:0/2130238429 conn(0x7f347c0b0a90 msgr2=0x7f347c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:14.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.534+0000 7f3483741640 1 -- 172.21.15.72:0/2130238429 shutdown_connections 2023-12-12T09:55:14.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.534+0000 7f3483741640 1 -- 172.21.15.72:0/2130238429 wait complete. 2023-12-12T09:55:14.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.535+0000 7f3483741640 1 Processor -- start 2023-12-12T09:55:14.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.535+0000 7f3483741640 1 -- start start 2023-12-12T09:55:14.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.536+0000 7f3483741640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c06de20 0x7f347c0af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.536+0000 7f3483741640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c157a60 0x7f347c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.536+0000 7f3483741640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.536+0000 7f3483741640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c15b880 con 0x7f347c15e2b0 2023-12-12T09:55:14.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.536+0000 7f3483741640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c1461b0 con 0x7f347c06de20 2023-12-12T09:55:14.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.536+0000 7f3483741640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0af6b0 con 0x7f347c157a60 2023-12-12T09:55:14.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37120/0 (socket says 172.21.15.72:37120) 2023-12-12T09:55:14.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 -- 172.21.15.72:0/2641341432 learned_addr learned my addr 172.21.15.72:0/2641341432 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:14.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f34814dd640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c06de20 0x7f347c0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 -- 172.21.15.72:0/2641341432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c157a60 msgr2=0x7f347c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c157a60 0x7f347c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 -- 172.21.15.72:0/2641341432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c06de20 msgr2=0x7f347c0af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c06de20 0x7f347c0af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 -- 172.21.15.72:0/2641341432 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347400d410 con 0x7f347c15e2b0 2023-12-12T09:55:14.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.537+0000 7f3481cde640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cccc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f347400d8a0 tx=0x7f34740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.538+0000 7f3479ffb640 1 -- 172.21.15.72:0/2641341432 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474041640 con 0x7f347c15e2b0 2023-12-12T09:55:14.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.538+0000 7f3479ffb640 1 -- 172.21.15.72:0/2641341432 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34740417a0 con 0x7f347c15e2b0 2023-12-12T09:55:14.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.538+0000 7f3479ffb640 1 -- 172.21.15.72:0/2641341432 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474041b50 con 0x7f347c15e2b0 2023-12-12T09:55:14.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.538+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c0b0420 con 0x7f347c15e2b0 2023-12-12T09:55:14.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.538+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c15d880 con 0x7f347c15e2b0 2023-12-12T09:55:14.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.539+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c15b9c0 con 0x7f347c15e2b0 2023-12-12T09:55:14.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.540+0000 7f3479ffb640 1 -- 172.21.15.72:0/2641341432 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f347403d040 con 0x7f347c15e2b0 2023-12-12T09:55:14.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.541+0000 7f3479ffb640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f346006ae40 0x7f346006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.541+0000 7f3479ffb640 1 -- 172.21.15.72:0/2641341432 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f34740ad8f0 con 0x7f347c15e2b0 2023-12-12T09:55:14.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.541+0000 7f34814dd640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f346006ae40 0x7f346006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.542+0000 7f34814dd640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f346006ae40 0x7f346006d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3470009fd0 tx=0x7f347000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.546+0000 7f3479ffb640 1 -- 172.21.15.72:0/2641341432 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f347c15b9c0 con 0x7f347c15e2b0 2023-12-12T09:55:14.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.737+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f347c06e2b0 con 0x7f346006ae40 2023-12-12T09:55:14.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.738+0000 7f3479ffb640 1 -- 172.21.15.72:0/2641341432 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3470016420 con 0x7f346006ae40 2023-12-12T09:55:14.740 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 6 to deep-scrub 2023-12-12T09:55:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.740+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f346006ae40 msgr2=0x7f346006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.740+0000 7f3483741640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f346006ae40 0x7f346006d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3470009fd0 tx=0x7f347000b040 comp rx=0 tx=0).stop 2023-12-12T09:55:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.740+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 msgr2=0x7f347c0cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.740+0000 7f3483741640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cccc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f347400d8a0 tx=0x7f34740046c0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.741+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 shutdown_connections 2023-12-12T09:55:14.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.741+0000 7f3483741640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f346006ae40 0x7f346006d300 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.741+0000 7f3483741640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f347c15e2b0 0x7f347c0cccc0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.741+0000 7f3483741640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c157a60 0x7f347c15d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.741+0000 7f3483741640 1 --2- 172.21.15.72:0/2641341432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c06de20 0x7f347c0af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.741+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 >> 172.21.15.72:0/2641341432 conn(0x7f347c0b0a90 msgr2=0x7f347c0cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:14.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.741+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 shutdown_connections 2023-12-12T09:55:14.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.741+0000 7f3483741640 1 -- 172.21.15.72:0/2641341432 wait complete. 2023-12-12T09:55:14.760 INFO:tasks.ceph:Scrubbing osd.7 2023-12-12T09:55:14.760 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-12T09:55:14.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.974+0000 7f02278ad640 1 Processor -- start 2023-12-12T09:55:14.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.974+0000 7f02278ad640 1 -- start start 2023-12-12T09:55:14.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f02278ad640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02201448a0 0x7f022006c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f02278ad640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f022015ec10 0x7f022015f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f02278ad640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022015f5a0 con 0x7f02200724a0 2023-12-12T09:55:14.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f02278ad640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02200b3c50 con 0x7f022015ec10 2023-12-12T09:55:14.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f02278ad640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b3960 con 0x7f02201448a0 2023-12-12T09:55:14.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f0224e48640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f02200724a0 0x7f02200b1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43152/0 (socket says 172.21.15.72:43152) 2023-12-12T09:55:14.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f0224e48640 1 -- 172.21.15.72:0/2926441377 learned_addr learned my addr 172.21.15.72:0/2926441377 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:14.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f0225e4a640 1 --2- 172.21.15.72:0/2926441377 >> v2:172.21.15.72:3300/0 conn(0x7f022015ec10 0x7f022015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.976+0000 7f0225649640 1 --2- 172.21.15.72:0/2926441377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02201448a0 0x7f022006c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.977+0000 7f0217fff640 1 -- 172.21.15.72:0/2926441377 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37452582 0 0) 0x7f0210001110 con 0x7f02200724a0 2023-12-12T09:55:14.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.977+0000 7f0225e4a640 1 -- 172.21.15.72:0/2926441377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02201448a0 msgr2=0x7f022006c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:14.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.977+0000 7f0225e4a640 1 --2- 172.21.15.72:0/2926441377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02201448a0 0x7f022006c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.977+0000 7f0225e4a640 1 -- 172.21.15.72:0/2926441377 >> v1:172.21.15.72:6789/0 conn(0x7f02200724a0 legacy=0x7f02200b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.977+0000 7f0225e4a640 1 -- 172.21.15.72:0/2926441377 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0220071140 con 0x7f022015ec10 2023-12-12T09:55:14.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.977+0000 7f0225649640 1 --2- 172.21.15.72:0/2926441377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02201448a0 0x7f022006c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:55:14.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.978+0000 7f0225e4a640 1 --2- 172.21.15.72:0/2926441377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f022015ec10 0x7f022015f060 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0218009d90 tx=0x7f021802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191ef0f3183e5aa9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.978+0000 7f0217fff640 1 -- 172.21.15.72:0/2926441377 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021802cc50 con 0x7f022015ec10 2023-12-12T09:55:14.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.978+0000 7f0217fff640 1 -- 172.21.15.72:0/2926441377 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021802cdb0 con 0x7f022015ec10 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.978+0000 7f0217fff640 1 -- 172.21.15.72:0/2926441377 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218034660 con 0x7f022015ec10 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.979+0000 7f02278ad640 1 -- 172.21.15.72:0/2926441377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f022015ec10 msgr2=0x7f022015f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.979+0000 7f02278ad640 1 --2- 172.21.15.72:0/2926441377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f022015ec10 0x7f022015f060 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0218009d90 tx=0x7f021802bea0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.979+0000 7f02278ad640 1 -- 172.21.15.72:0/2926441377 shutdown_connections 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.979+0000 7f02278ad640 1 --2- 172.21.15.72:0/2926441377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f022015ec10 0x7f022015f060 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.979+0000 7f02278ad640 1 --2- 172.21.15.72:0/2926441377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02201448a0 0x7f022006c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.979+0000 7f02278ad640 1 -- 172.21.15.72:0/2926441377 >> 172.21.15.72:0/2926441377 conn(0x7f02200b0a90 msgr2=0x7f02200b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.979+0000 7f02278ad640 1 -- 172.21.15.72:0/2926441377 shutdown_connections 2023-12-12T09:55:14.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.979+0000 7f02278ad640 1 -- 172.21.15.72:0/2926441377 wait complete. 2023-12-12T09:55:14.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.981+0000 7f02278ad640 1 Processor -- start 2023-12-12T09:55:14.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.981+0000 7f02278ad640 1 -- start start 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f02278ad640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02200724a0 0x7f022013e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f02278ad640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02201448a0 0x7f02200cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f02278ad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f022015ec10 0x7f02200cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f02278ad640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022015ab50 con 0x7f02201448a0 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f02278ad640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200cd4b0 con 0x7f022015ec10 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f02278ad640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200cd790 con 0x7f02200724a0 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f0224e48640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02201448a0 0x7f02200cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f0224e48640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02201448a0 0x7f02200cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37134/0 (socket says 172.21.15.72:37134) 2023-12-12T09:55:14.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.982+0000 7f0224e48640 1 -- 172.21.15.72:0/89659711 learned_addr learned my addr 172.21.15.72:0/89659711 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:14.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0225649640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02200724a0 0x7f022013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0225e4a640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f022015ec10 0x7f02200cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0224e48640 1 -- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02200724a0 msgr2=0x7f022013e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0224e48640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02200724a0 0x7f022013e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0224e48640 1 -- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f022015ec10 msgr2=0x7f02200cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:14.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0224e48640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f022015ec10 0x7f02200cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:14.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0224e48640 1 -- 172.21.15.72:0/89659711 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218009a70 con 0x7f02201448a0 2023-12-12T09:55:14.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0224e48640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02201448a0 0x7f02200cc8f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f021000c430 tx=0x7f021000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02100074e0 con 0x7f02201448a0 2023-12-12T09:55:14.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210007660 con 0x7f02201448a0 2023-12-12T09:55:14.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210004a20 con 0x7f02201448a0 2023-12-12T09:55:14.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02200cda70 con 0x7f02201448a0 2023-12-12T09:55:14.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.983+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02200ce030 con 0x7f02201448a0 2023-12-12T09:55:14.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.984+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02200b24a0 con 0x7f02201448a0 2023-12-12T09:55:14.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.985+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0210004b80 con 0x7f02201448a0 2023-12-12T09:55:14.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.986+0000 7f0215ffb640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f020406ae40 0x7f020406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.986+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0210089160 con 0x7f02201448a0 2023-12-12T09:55:14.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.987+0000 7f0225649640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f020406ae40 0x7f020406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.987+0000 7f0215ffb640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f0204071140 0x7f0204073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:14.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.987+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 --> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0204073c50 con 0x7f0204071140 2023-12-12T09:55:14.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.987+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f021008b820 con 0x7f02201448a0 2023-12-12T09:55:14.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.987+0000 7f0225649640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f020406ae40 0x7f020406d300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f021c006fd0 tx=0x7f021c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.987+0000 7f0225e4a640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f0204071140 0x7f0204073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:14.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.988+0000 7f0225e4a640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f0204071140 0x7f0204073580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:14.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:14.990+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 <== osd.7 v2:172.21.15.113:6824/1904806807 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0204073c50 con 0x7f0204071140 2023-12-12T09:55:15.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.005+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 --> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0220071700 con 0x7f0204071140 2023-12-12T09:55:15.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.005+0000 7f0215ffb640 1 -- 172.21.15.72:0/89659711 <== osd.7 v2:172.21.15.113:6824/1904806807 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0218009450 con 0x7f0204071140 2023-12-12T09:55:15.008 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-12-12T09:55:15.008 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-12T09:55:15.008 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-12-12T09:55:15.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f0204071140 msgr2=0x7f0204073580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.113:6824/1904806807,v1:172.21.15.113:6825/1904806807] conn(0x7f0204071140 0x7f0204073580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f020406ae40 msgr2=0x7f020406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f020406ae40 0x7f020406d300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f021c006fd0 tx=0x7f021c008040 comp rx=0 tx=0).stop 2023-12-12T09:55:15.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02201448a0 msgr2=0x7f02200cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 --2- 172.21.15.72:0/89659711 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02201448a0 0x7f02200cc8f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f021000c430 tx=0x7f021000c8f0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f0225e4a640 1 -- 172.21.15.72:0/89659711 reap_dead start 2023-12-12T09:55:15.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 shutdown_connections 2023-12-12T09:55:15.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 >> 172.21.15.72:0/89659711 conn(0x7f02200b0a90 msgr2=0x7f022006d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:15.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.006+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 shutdown_connections 2023-12-12T09:55:15.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.007+0000 7f02278ad640 1 -- 172.21.15.72:0/89659711 wait complete. 2023-12-12T09:55:15.028 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-12T09:55:15.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.239+0000 7f61197f4640 1 Processor -- start 2023-12-12T09:55:15.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.240+0000 7f61197f4640 1 -- start start 2023-12-12T09:55:15.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.241+0000 7f61197f4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f611406dd00 0x7f6114071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.241+0000 7f61197f4640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61140abd20 con 0x7f611415d150 2023-12-12T09:55:15.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.241+0000 7f61197f4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611415d530 con 0x7f6114070780 2023-12-12T09:55:15.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.241+0000 7f61197f4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61140b2950 con 0x7f611406dd00 2023-12-12T09:55:15.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.242+0000 7f6113fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f611415d150 0x7f6114070070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47164/0 (socket says 172.21.15.72:47164) 2023-12-12T09:55:15.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.242+0000 7f6113fff640 1 -- 172.21.15.72:0/937704888 learned_addr learned my addr 172.21.15.72:0/937704888 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:15.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.242+0000 7f6118ff3640 1 --2- 172.21.15.72:0/937704888 >> v2:172.21.15.113:3300/0 conn(0x7f611406dd00 0x7f6114071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.242+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926077811 0 0) 0x7f61140abd20 con 0x7f611415d150 2023-12-12T09:55:15.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.242+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6100003520 con 0x7f611415d150 2023-12-12T09:55:15.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2667744830 0 0) 0x7f6100003520 con 0x7f611415d150 2023-12-12T09:55:15.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 >> v2:172.21.15.113:3300/0 conn(0x7f611406dd00 msgr2=0x7f6114071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:15.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 --2- 172.21.15.72:0/937704888 >> v2:172.21.15.113:3300/0 conn(0x7f611406dd00 0x7f6114071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 >> v1:172.21.15.113:6789/0 conn(0x7f6114070780 legacy=0x7f611406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61140b2c30 con 0x7f611415d150 2023-12-12T09:55:15.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3524263480 0 0) 0x7f6104001670 con 0x7f611415d150 2023-12-12T09:55:15.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 >> v1:172.21.15.72:6789/0 conn(0x7f611415d150 legacy=0x7f6114070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000039c0 0x7f6100003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61000047c0 0x7f6100004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6100005180 0x7f610000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61140b2950 con 0x7f61000047c0 2023-12-12T09:55:15.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610000c080 con 0x7f61000039c0 2023-12-12T09:55:15.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610000c240 con 0x7f6100005180 2023-12-12T09:55:15.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.243+0000 7f610bfff640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61000047c0 0x7f6100004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f6113fff640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000039c0 0x7f6100003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f6118ff3640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6100005180 0x7f610000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f610bfff640 1 -- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6100005180 msgr2=0x7f610000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f610bfff640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6100005180 0x7f610000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f610bfff640 1 -- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000039c0 msgr2=0x7f6100003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f610bfff640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000039c0 0x7f6100003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f610bfff640 1 -- 172.21.15.72:0/937704888 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6100003520 con 0x7f61000047c0 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f6113fff640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000039c0 0x7f6100003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:15.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.244+0000 7f6118ff3640 1 -- 172.21.15.72:0/937704888 reap_dead start 2023-12-12T09:55:15.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.245+0000 7f610bfff640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61000047c0 0x7f6100004c40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f60fc001750 tx=0x7f61000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909c2df3b8ab42c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:15.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.245+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc0052d0 con 0x7f61000047c0 2023-12-12T09:55:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.245+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc02e050 con 0x7f61000047c0 2023-12-12T09:55:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.245+0000 7f61137fe640 1 -- 172.21.15.72:0/937704888 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc003830 con 0x7f61000047c0 2023-12-12T09:55:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.246+0000 7f61197f4640 1 -- 172.21.15.72:0/937704888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61000047c0 msgr2=0x7f6100004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.246+0000 7f61197f4640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61000047c0 0x7f6100004c40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f60fc001750 tx=0x7f61000057f0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.246+0000 7f61197f4640 1 -- 172.21.15.72:0/937704888 shutdown_connections 2023-12-12T09:55:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.246+0000 7f61197f4640 1 --2- 172.21.15.72:0/937704888 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61000047c0 0x7f6100004c40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.246+0000 7f61197f4640 1 -- 172.21.15.72:0/937704888 >> 172.21.15.72:0/937704888 conn(0x7f61140b0860 msgr2=0x7f61140b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:15.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.246+0000 7f61197f4640 1 -- 172.21.15.72:0/937704888 shutdown_connections 2023-12-12T09:55:15.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.246+0000 7f61197f4640 1 -- 172.21.15.72:0/937704888 wait complete. 2023-12-12T09:55:15.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.248+0000 7f61197f4640 1 Processor -- start 2023-12-12T09:55:15.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.248+0000 7f61197f4640 1 -- start start 2023-12-12T09:55:15.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f61197f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000047c0 0x7f6114158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f61197f4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61141f37b0 0x7f611415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f61197f4640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f611415f250 0x7f61141f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f61197f4640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61141f7080 con 0x7f611415f250 2023-12-12T09:55:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f61197f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611415f6d0 con 0x7f61000047c0 2023-12-12T09:55:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f61197f4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61141f3320 con 0x7f61141f37b0 2023-12-12T09:55:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f6118ff3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f611415f250 0x7f61141f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f6118ff3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f611415f250 0x7f61141f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54228/0 (socket says 172.21.15.72:54228) 2023-12-12T09:55:15.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.249+0000 7f6118ff3640 1 -- 172.21.15.72:0/2717623859 learned_addr learned my addr 172.21.15.72:0/2717623859 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f6113fff640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000047c0 0x7f6114158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f610bfff640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61141f37b0 0x7f611415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f6118ff3640 1 -- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61141f37b0 msgr2=0x7f611415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f6118ff3640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61141f37b0 0x7f611415ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f6118ff3640 1 -- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000047c0 msgr2=0x7f6114158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f6118ff3640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000047c0 0x7f6114158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f6118ff3640 1 -- 172.21.15.72:0/2717623859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114158b90 con 0x7f611415f250 2023-12-12T09:55:15.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f610bfff640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61141f37b0 0x7f611415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:15.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f6118ff3640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f611415f250 0x7f61141f60b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f611406e500 tx=0x7f610c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f61117fa640 1 -- 172.21.15.72:0/2717623859 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c016420 con 0x7f611415f250 2023-12-12T09:55:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f61117fa640 1 -- 172.21.15.72:0/2717623859 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c0165a0 con 0x7f611415f250 2023-12-12T09:55:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f61117fa640 1 -- 172.21.15.72:0/2717623859 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c016950 con 0x7f611415f250 2023-12-12T09:55:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61141f2a60 con 0x7f611415f250 2023-12-12T09:55:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.250+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61141f2cd0 con 0x7f611415f250 2023-12-12T09:55:15.259 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.252+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6114074450 con 0x7f611415f250 2023-12-12T09:55:15.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.253+0000 7f61117fa640 1 -- 172.21.15.72:0/2717623859 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f610c016ab0 con 0x7f611415f250 2023-12-12T09:55:15.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.254+0000 7f61117fa640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60f406ae40 0x7f60f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.254+0000 7f61117fa640 1 -- 172.21.15.72:0/2717623859 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f610c08a520 con 0x7f611415f250 2023-12-12T09:55:15.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.254+0000 7f6113fff640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60f406ae40 0x7f60f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.255+0000 7f6113fff640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60f406ae40 0x7f60f406d300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f611409cf20 tx=0x7f6104009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:15.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.258+0000 7f61117fa640 1 -- 172.21.15.72:0/2717623859 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f610c055770 con 0x7f611415f250 2023-12-12T09:55:15.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.447+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f611406c170 con 0x7f60f406ae40 2023-12-12T09:55:15.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.448+0000 7f61117fa640 1 -- 172.21.15.72:0/2717623859 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f610400d050 con 0x7f60f406ae40 2023-12-12T09:55:15.450 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 7 to deep-scrub 2023-12-12T09:55:15.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60f406ae40 msgr2=0x7f60f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60f406ae40 0x7f60f406d300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f611409cf20 tx=0x7f6104009040 comp rx=0 tx=0).stop 2023-12-12T09:55:15.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f611415f250 msgr2=0x7f61141f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f611415f250 0x7f61141f60b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f611406e500 tx=0x7f610c00f520 comp rx=0 tx=0).stop 2023-12-12T09:55:15.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 shutdown_connections 2023-12-12T09:55:15.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f60f406ae40 0x7f60f406d300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f611415f250 0x7f61141f60b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61141f37b0 0x7f611415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 --2- 172.21.15.72:0/2717623859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61000047c0 0x7f6114158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 >> 172.21.15.72:0/2717623859 conn(0x7f61140b0860 msgr2=0x7f61140ac400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:15.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 shutdown_connections 2023-12-12T09:55:15.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.451+0000 7f61197f4640 1 -- 172.21.15.72:0/2717623859 wait complete. 2023-12-12T09:55:15.472 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:55:15.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.676+0000 7f2dea456640 1 Processor -- start 2023-12-12T09:55:15.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.676+0000 7f2dea456640 1 -- start start 2023-12-12T09:55:15.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2dea456640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de40708d0 0x7f2de40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2dea456640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2de406d520 0x7f2de41626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2dea456640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de40cde40 con 0x7f2de415f260 2023-12-12T09:55:15.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2dea456640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de40cd040 con 0x7f2de406d520 2023-12-12T09:55:15.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2dea456640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4162bf0 con 0x7f2de40708d0 2023-12-12T09:55:15.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2de89f3640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2de406d520 0x7f2de41626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2de3fff640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f2de415f260 0x7f2de40701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:47172/0 (socket says 172.21.15.72:47172) 2023-12-12T09:55:15.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2de3fff640 1 -- 172.21.15.72:0/573752100 learned_addr learned my addr 172.21.15.72:0/573752100 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:15.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.678+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/573752100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de40708d0 0x7f2de40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.679+0000 7f2de89f3640 1 -- 172.21.15.72:0/573752100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de40708d0 msgr2=0x7f2de40ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:15.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.679+0000 7f2de89f3640 1 --2- 172.21.15.72:0/573752100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de40708d0 0x7f2de40ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.679+0000 7f2de89f3640 1 -- 172.21.15.72:0/573752100 >> v1:172.21.15.72:6789/0 conn(0x7f2de415f260 legacy=0x7f2de40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.679+0000 7f2de89f3640 1 -- 172.21.15.72:0/573752100 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40cd230 con 0x7f2de406d520 2023-12-12T09:55:15.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.679+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/573752100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de40708d0 0x7f2de40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:55:15.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.680+0000 7f2de89f3640 1 --2- 172.21.15.72:0/573752100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de406d520 0x7f2de41626b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009d90 tx=0x7f2ddc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1d8ccbf59a8837 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:15.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.680+0000 7f2de37fe640 1 -- 172.21.15.72:0/573752100 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc02cc60 con 0x7f2de406d520 2023-12-12T09:55:15.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.680+0000 7f2de37fe640 1 -- 172.21.15.72:0/573752100 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc02cdc0 con 0x7f2de406d520 2023-12-12T09:55:15.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.680+0000 7f2de37fe640 1 -- 172.21.15.72:0/573752100 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc034660 con 0x7f2de406d520 2023-12-12T09:55:15.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.681+0000 7f2dea456640 1 -- 172.21.15.72:0/573752100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de406d520 msgr2=0x7f2de41626b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.681+0000 7f2dea456640 1 --2- 172.21.15.72:0/573752100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de406d520 0x7f2de41626b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009d90 tx=0x7f2ddc02beb0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.681+0000 7f2dea456640 1 -- 172.21.15.72:0/573752100 shutdown_connections 2023-12-12T09:55:15.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.681+0000 7f2dea456640 1 --2- 172.21.15.72:0/573752100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de406d520 0x7f2de41626b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.681+0000 7f2dea456640 1 --2- 172.21.15.72:0/573752100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de40708d0 0x7f2de40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.681+0000 7f2dea456640 1 -- 172.21.15.72:0/573752100 >> 172.21.15.72:0/573752100 conn(0x7f2de40b0a90 msgr2=0x7f2de40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:15.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.681+0000 7f2dea456640 1 -- 172.21.15.72:0/573752100 shutdown_connections 2023-12-12T09:55:15.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.681+0000 7f2dea456640 1 -- 172.21.15.72:0/573752100 wait complete. 2023-12-12T09:55:15.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.683+0000 7f2dea456640 1 Processor -- start 2023-12-12T09:55:15.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.683+0000 7f2dea456640 1 -- start start 2023-12-12T09:55:15.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.684+0000 7f2dea456640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de406d520 0x7f2de40abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.684+0000 7f2dea456640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de40708d0 0x7f2de415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.684+0000 7f2dea456640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de415f260 0x7f2de40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.684+0000 7f2dea456640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de41fb250 con 0x7f2de40708d0 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.684+0000 7f2dea456640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40a8450 con 0x7f2de406d520 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.684+0000 7f2dea456640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40a8730 con 0x7f2de415f260 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de40708d0 0x7f2de415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de40708d0 0x7f2de415afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54254/0 (socket says 172.21.15.72:54254) 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/3345151891 learned_addr learned my addr 172.21.15.72:0/3345151891 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/3345151891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de415f260 msgr2=0x7f2de40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2de89f3640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de415f260 0x7f2de40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2de3fff640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de406d520 0x7f2de40abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de415f260 0x7f2de40b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/3345151891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de406d520 msgr2=0x7f2de40abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de406d520 0x7f2de40abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 -- 172.21.15.72:0/3345151891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc009a70 con 0x7f2de40708d0 2023-12-12T09:55:15.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.685+0000 7f2ddb7fe640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de40708d0 0x7f2de415afe0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00e7e0 tx=0x7f2dcc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:15.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.686+0000 7f2de17fa640 1 -- 172.21.15.72:0/3345151891 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc004030 con 0x7f2de40708d0 2023-12-12T09:55:15.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.686+0000 7f2de17fa640 1 -- 172.21.15.72:0/3345151891 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc004190 con 0x7f2de40708d0 2023-12-12T09:55:15.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.686+0000 7f2de17fa640 1 -- 172.21.15.72:0/3345151891 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc004520 con 0x7f2de40708d0 2023-12-12T09:55:15.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.686+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de40a8a10 con 0x7f2de40708d0 2023-12-12T09:55:15.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.686+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de40a8fd0 con 0x7f2de40708d0 2023-12-12T09:55:15.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.687+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de40745d0 con 0x7f2de40708d0 2023-12-12T09:55:15.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.688+0000 7f2de17fa640 1 -- 172.21.15.72:0/3345151891 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2dcc00c6d0 con 0x7f2de40708d0 2023-12-12T09:55:15.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.689+0000 7f2de17fa640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dc406ae40 0x7f2dc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.689+0000 7f2de17fa640 1 -- 172.21.15.72:0/3345151891 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2dcc089060 con 0x7f2de40708d0 2023-12-12T09:55:15.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.689+0000 7f2de3fff640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dc406ae40 0x7f2dc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.690+0000 7f2de3fff640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dc406ae40 0x7f2dc406d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4005880 tx=0x7f2dd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:15.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.693+0000 7f2de17fa640 1 -- 172.21.15.72:0/3345151891 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2dcc054360 con 0x7f2de40708d0 2023-12-12T09:55:15.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.878+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2de40cd7f0 con 0x7f2dc406ae40 2023-12-12T09:55:15.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.881+0000 7f2de17fa640 1 -- 172.21.15.72:0/3345151891 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48266 (secure 0 0 0) 0x7f2dd4001350 con 0x7f2dc406ae40 2023-12-12T09:55:15.884 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:55:15.885 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:55:15.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.884+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dc406ae40 msgr2=0x7f2dc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.884+0000 7f2dea456640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dc406ae40 0x7f2dc406d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4005880 tx=0x7f2dd4009040 comp rx=0 tx=0).stop 2023-12-12T09:55:15.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.884+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de40708d0 msgr2=0x7f2de415afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.884+0000 7f2dea456640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de40708d0 0x7f2de415afe0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00e7e0 tx=0x7f2dcc00eca0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.885+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 shutdown_connections 2023-12-12T09:55:15.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.885+0000 7f2dea456640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f2dc406ae40 0x7f2dc406d300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.885+0000 7f2dea456640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de415f260 0x7f2de40b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.885+0000 7f2dea456640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2de40708d0 0x7f2de415afe0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.885+0000 7f2dea456640 1 --2- 172.21.15.72:0/3345151891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de406d520 0x7f2de40abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.885+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 >> 172.21.15.72:0/3345151891 conn(0x7f2de40b0a90 msgr2=0x7f2de4162dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:15.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.885+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 shutdown_connections 2023-12-12T09:55:15.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:15.885+0000 7f2dea456640 1 -- 172.21.15.72:0/3345151891 wait complete. 2023-12-12T09:55:15.904 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2023-12-12T09:55:15.604174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255880,"kb_used_data":6428,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718840,"statfs":{"total":773094113280,"available":772832092160,"internally_reserved":0,"allocated":6582272,"data_stored":3571719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003062"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":197,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-12-12T09:55:02.535538+0000","last_change":"2023-12-12T09:52:03.160616+0000","last_active":"2023-12-12T09:55:02.535538+0000","last_peered":"2023-12-12T09:55:02.535538+0000","last_clean":"2023-12-12T09:55:02.535538+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:55:02.535538+0000","last_undegraded":"2023-12-12T09:55:02.535538+0000","last_fullsized":"2023-12-12T09:55:02.535538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:01.788956+0000","last_clean_scrub_stamp":"2023-12-12T09:52:01.788956+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:30:58.425993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1269760,"data_stored":889623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0880000000000001,"15min":1.0900000000000001},"min":{"1min":0.61899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6799999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1559999999999999,"15min":1.1419999999999999},"min":{"1min":1.0089999999999999,"5min":0.622,"15min":0.622},"max":{"1min":1.6950000000000001,"5min":1.855,"15min":1.855},"last":1.393}]},{"osd":1,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0369999999999999,"15min":1.024},"min":{"1min":0.75800000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5189999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.878}]},{"osd":2,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3280000000000001,"15min":1.3819999999999999},"min":{"1min":0.83499999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.8420000000000001,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.369,"15min":1.448},"min":{"1min":0.877,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5589999999999999,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.2869999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3520000000000001,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8899999999999999,"5min":2.23,"15min":2.23},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.4079999999999999,"15min":1.4179999999999999},"min":{"1min":0.91800000000000004,"5min":0.746,"15min":0.746},"max":{"1min":1.845,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.171}]},{"osd":4,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3939999999999999,"15min":1.4299999999999999},"min":{"1min":0.78600000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.76,"5min":2.649,"15min":2.649},"last":1.504},{"interface":"front","average":{"1min":1.329,"5min":1.3859999999999999,"15min":1.421},"min":{"1min":0.89600000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7749999999999999,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.454}]},{"osd":5,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.498,"15min":1.556},"min":{"1min":0.93999999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.766,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.5449999999999999,"15min":1.5620000000000001},"min":{"1min":1.0009999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.901,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.647}]},{"osd":6,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.5880000000000001,"15min":1.6180000000000001},"min":{"1min":1.1850000000000001,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.921,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.546},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.571,"15min":1.617},"min":{"1min":1.282,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.736,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.5740000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0529999999999999,"15min":1.0449999999999999},"min":{"1min":0.76000000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5820000000000001,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.998},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.115,"15min":1.107},"min":{"1min":0.754,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.702,"5min":1.865,"15min":1.865},"last":1.6240000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2330000000000001,"15min":1.2270000000000001},"min":{"1min":0.81999999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7250000000000001,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.95699999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.5940000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.5609999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.139,"15min":1.133},"min":{"1min":0.747,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.393,"5min":1.859,"15min":1.859},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2769999999999999,"15min":1.2889999999999999},"min":{"1min":0.754,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.5589999999999999,"5min":2.028,"15min":2.028},"last":0.86599999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3200000000000001,"15min":1.3109999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.26,"15min":1.256},"min":{"1min":0.90900000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7210000000000001,"5min":1.994,"15min":1.994},"last":1.0509999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.379,"15min":1.393},"min":{"1min":0.92000000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6799999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.5189999999999999,"15min":1.5369999999999999},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.1269999999999998,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.395}]},{"osd":5,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3819999999999999,"15min":1.377},"min":{"1min":1.0229999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.6679999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.996,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":1.7829999999999999,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.2230000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.6619999999999999,"15min":1.696},"min":{"1min":0.93999999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2269999999999999,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.5189999999999999,"15min":1.5129999999999999},"min":{"1min":1.288,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8200000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.4410000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.091,"15min":1.042},"min":{"1min":0.89800000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.823,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.196},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.194,"15min":1.24},"min":{"1min":0.68000000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.734,"5min":3.694,"15min":3.694},"last":0.90600000000000003}]},{"osd":1,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.2689999999999999,"15min":1.331},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7230000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.486},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0820000000000001,"15min":1.0940000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.5069999999999999,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.97899999999999998}]},{"osd":2,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1240000000000001,"15min":1.087},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.242},{"interface":"front","average":{"1min":1.234,"5min":1.145,"15min":1.1040000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7470000000000001,"5min":1.895,"15min":1.895},"last":1.3049999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.351,"15min":1.367},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8620000000000001,"5min":3.944,"15min":3.944},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.3149999999999999,"15min":1.2989999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.4419999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3460000000000001,"15min":1.337},"min":{"1min":0.80300000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.962,"5min":2.536,"15min":2.536},"last":1.071},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4910000000000001,"15min":1.5409999999999999},"min":{"1min":0.82999999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.218,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.6040000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.446,"15min":1.4590000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.1669999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.494,"15min":1.496},"min":{"1min":0.94799999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.0990000000000002,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.395}]},{"osd":7,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.5760000000000001,"15min":1.6180000000000001},"min":{"1min":0.70699999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1339999999999999,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.6140000000000001,"15min":1.653},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4350000000000001,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.353}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":298236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.101,"15min":1.079},"min":{"1min":0.626,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.639,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.224,"15min":1.1259999999999999},"min":{"1min":0.745,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7410000000000001,"5min":7.29,"15min":7.29},"last":0.998}]},{"osd":1,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2529999999999999,"15min":1.179},"min":{"1min":0.91700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.587,"5min":7.3360000000000003,"15min":7.3360000000000003},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.282,"15min":1.284},"min":{"1min":0.88900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":6.7530000000000001,"15min":6.7530000000000001},"last":1.446}]},{"osd":2,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.165,"15min":1.1539999999999999},"min":{"1min":0.69899999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6339999999999999,"5min":4.891,"15min":4.891},"last":1.472},{"interface":"front","average":{"1min":1.294,"5min":1.375,"15min":1.3120000000000001},"min":{"1min":0.747,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8169999999999999,"5min":7.4100000000000001,"15min":7.4100000000000001},"last":1.292}]},{"osd":3,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3200000000000001,"15min":1.306},"min":{"1min":0.81100000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.635,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":1.389},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.323,"15min":1.3420000000000001},"min":{"1min":0.74099999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0720000000000001,"5min":4.7409999999999997,"15min":4.7409999999999997},"last":1.3600000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.4970000000000001,"15min":1.427},"min":{"1min":0.81399999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8899999999999999,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":1.569},{"interface":"front","average":{"1min":1.409,"5min":1.4930000000000001,"15min":1.4850000000000001},"min":{"1min":0.89600000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8959999999999999,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":0.93300000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.405,"15min":1.4039999999999999},"min":{"1min":0.86799999999999999,"5min":0.629,"15min":0.629},"max":{"1min":2.1880000000000002,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.141},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.6080000000000001,"15min":1.5580000000000001},"min":{"1min":0.95299999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9730000000000001,"5min":7.3739999999999997,"15min":7.3739999999999997},"last":1.5089999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.587,"15min":1.569},"min":{"1min":1.046,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.0099999999999998,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.6100000000000001,"15min":1.577},"min":{"1min":1.137,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.23,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.542}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82099999999999995,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4510000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93000000000000005,"15min":0.93999999999999995},"min":{"1min":0.629,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.206,"5min":1.577,"15min":1.577},"last":1.0469999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.873,"15min":0.871},"min":{"1min":0.55600000000000005,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1499999999999999,"5min":1.548,"15min":1.548},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.105,"5min":0.996,"15min":0.98199999999999998},"min":{"1min":0.66700000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.24}]},{"osd":2,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.048,"15min":1.0629999999999999},"min":{"1min":0.67500000000000004,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4710000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.504,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.377}]},{"osd":4,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.81599999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2360000000000002,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.46},{"interface":"front","average":{"1min":1.367,"5min":1.272,"15min":1.268},"min":{"1min":1.046,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.782,"5min":1.964,"15min":1.964},"last":1.1559999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3009999999999999,"15min":1.3169999999999999},"min":{"1min":0.97899999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7290000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.327,"15min":1.323},"min":{"1min":0.83699999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.748,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.198}]},{"osd":6,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.321,"15min":1.329},"min":{"1min":0.68200000000000005,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.919,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.573},{"interface":"front","average":{"1min":1.385,"5min":1.327,"15min":1.3300000000000001},"min":{"1min":0.99399999999999999,"5min":0.627,"15min":0.627},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.3400000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3580000000000001,"15min":1.3560000000000001},"min":{"1min":1.081,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8080000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.022,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.831,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.504}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.33500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4510000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.871,"15min":0.90300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.508,"5min":1.752,"15min":1.752},"last":0.81399999999999995}]},{"osd":1,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.90800000000000003,"15min":0.92400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.573,"5min":1.603,"15min":1.603},"last":1.087},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.96499999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4830000000000001,"5min":1.633,"15min":1.633},"last":1.1379999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90100000000000002,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.494,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.369},{"interface":"front","average":{"1min":1.202,"5min":1.157,"15min":1.1839999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.754,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.5249999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.296,"15min":1.321},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.6799999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2390000000000001,"15min":1.2549999999999999},"min":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5900000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.294}]},{"osd":5,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.177,"15min":1.208},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.573,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3,"15min":1.3200000000000001},"min":{"1min":0.72999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6359999999999999,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.4370000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.337,"15min":1.381},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.345,"15min":1.387},"min":{"1min":0.872,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6850000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.2669999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.4159999999999999,"15min":1.468},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.742,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.377,"15min":1.4219999999999999},"min":{"1min":0.878,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.79,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.1910000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":298236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.669},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6719999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.0669999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.88300000000000001,"15min":0.876},"min":{"1min":0.47599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.337,"5min":1.712,"15min":1.712},"last":1.131},{"interface":"front","average":{"1min":1.151,"5min":1.024,"15min":1.004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.5880000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0269999999999999,"15min":1.0389999999999999},"min":{"1min":0.61699999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.655,"5min":1.665,"15min":1.665},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.532,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.92600000000000005}]},{"osd":4,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1160000000000001,"15min":1.1299999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6000000000000001,"5min":2.056,"15min":2.056},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.3069999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.885,"5min":2.427,"15min":2.427},"last":1.2969999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3520000000000001,"15min":1.3620000000000001},"min":{"1min":0.85399999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8149999999999999,"5min":3.476,"15min":3.476},"last":1.454},{"interface":"front","average":{"1min":1.359,"5min":1.3300000000000001,"15min":1.3500000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.744,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.5469999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.405,"15min":1.425},"min":{"1min":0.89600000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9330000000000001,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.246},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.341,"15min":1.363},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.6299999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3839999999999999,"15min":1.3979999999999999},"min":{"1min":0.93799999999999994,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9950000000000001,"5min":3.052,"15min":3.052},"last":1.421},{"interface":"front","average":{"1min":1.403,"5min":1.427,"15min":1.4470000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8360000000000001,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":1.5129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1265664,"data_stored":888604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.82699999999999996,"15min":0.83899999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.526,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.92500000000000004,"15min":0.99299999999999999},"min":{"1min":0.41499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.1000000000000001,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":0.88300000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.96599999999999997,"15min":0.98899999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.5309999999999999,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0329999999999999,"15min":1.077},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.734,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":1.734}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.97899999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.556,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1359999999999999},"min":{"1min":0.495,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5609999999999999,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.4379999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.1850000000000001,"15min":1.1910000000000001},"min":{"1min":0.71799999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7789999999999999,"5min":2.738,"15min":2.738},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.302,"15min":1.347},"min":{"1min":0.69799999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6180000000000001,"5min":3.875,"15min":3.875},"last":1.2749999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2430000000000001,"15min":1.268},"min":{"1min":0.76400000000000001,"5min":0.755,"15min":0.755},"max":{"1min":1.6000000000000001,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2629999999999999,"15min":1.2869999999999999},"min":{"1min":0.66200000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.677,"5min":2.54,"15min":2.54},"last":1.677}]},{"osd":6,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.278,"15min":1.2789999999999999},"min":{"1min":0.753,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.732,"5min":2.117,"15min":2.117},"last":1.246},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.383,"15min":1.429},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.744,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":1.639}]},{"osd":7,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.359,"15min":1.379},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.806,"5min":2.931,"15min":2.931},"last":1.333},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.448,"15min":1.53},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.851,"5min":4.548,"15min":4.548},"last":1.851}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:55:15.908 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-12-12T09:52:01.788956+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=12, tm_hour=9, tm_min=52, tm_sec=1, tm_wday=1, tm_yday=346, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=12, tm_hour=9, tm_min=55, tm_sec=8, tm_wday=1, tm_yday=346, tm_isdst=0) 2023-12-12T09:55:15.908 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-12-12T09:55:34.893 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:55:34.904 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:55:35.909 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:55:36.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.129+0000 7f6b1fce7640 1 Processor -- start 2023-12-12T09:55:36.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.129+0000 7f6b1fce7640 1 -- start start 2023-12-12T09:55:36.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.130+0000 7f6b1fce7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b18157a60 0x7f6b1806c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.130+0000 7f6b1fce7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b180cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.130+0000 7f6b1fce7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1806ca40 con 0x7f6b1806de20 2023-12-12T09:55:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.131+0000 7f6b1fce7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b18071cd0 con 0x7f6b18157a60 2023-12-12T09:55:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.131+0000 7f6b1fce7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18070f10 con 0x7f6b1815e2b0 2023-12-12T09:55:36.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.131+0000 7f6b1e284640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b180cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.131+0000 7f6b1da83640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b18157a60 0x7f6b1806c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.131+0000 7f6b1e284640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b180cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.72:50994/0 (socket says 172.21.15.72:50994) 2023-12-12T09:55:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.131+0000 7f6b1e284640 1 -- 172.21.15.72:0/3918238603 learned_addr learned my addr 172.21.15.72:0/3918238603 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.132+0000 7f6b1e284640 1 -- 172.21.15.72:0/3918238603 >> v2:172.21.15.113:3300/0 conn(0x7f6b18157a60 msgr2=0x7f6b1806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.132+0000 7f6b1e284640 1 --2- 172.21.15.72:0/3918238603 >> v2:172.21.15.113:3300/0 conn(0x7f6b18157a60 0x7f6b1806c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.132+0000 7f6b1e284640 1 -- 172.21.15.72:0/3918238603 >> v1:172.21.15.113:6789/0 conn(0x7f6b1806de20 legacy=0x7f6b1815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.132+0000 7f6b1e284640 1 -- 172.21.15.72:0/3918238603 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b180711f0 con 0x7f6b1815e2b0 2023-12-12T09:55:36.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.134+0000 7f6b1e284640 1 --2- 172.21.15.72:0/3918238603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b180cc8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009e50 tx=0x7f6b1002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd06073db976a799 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:36.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.134+0000 7f6b1ca81640 1 -- 172.21.15.72:0/3918238603 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1002ca70 con 0x7f6b1815e2b0 2023-12-12T09:55:36.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.134+0000 7f6b1ca81640 1 -- 172.21.15.72:0/3918238603 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1002cbd0 con 0x7f6b1815e2b0 2023-12-12T09:55:36.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.134+0000 7f6b1ca81640 1 -- 172.21.15.72:0/3918238603 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100343f0 con 0x7f6b1815e2b0 2023-12-12T09:55:36.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.135+0000 7f6b1fce7640 1 -- 172.21.15.72:0/3918238603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 msgr2=0x7f6b180cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.135+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/3918238603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b180cc8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009e50 tx=0x7f6b1002bed0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.135+0000 7f6b1fce7640 1 -- 172.21.15.72:0/3918238603 shutdown_connections 2023-12-12T09:55:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.135+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/3918238603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b180cc8f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.135+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/3918238603 >> v2:172.21.15.113:3300/0 conn(0x7f6b18157a60 0x7f6b1806c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.135+0000 7f6b1fce7640 1 -- 172.21.15.72:0/3918238603 >> 172.21.15.72:0/3918238603 conn(0x7f6b180b0a90 msgr2=0x7f6b180b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.135+0000 7f6b1fce7640 1 -- 172.21.15.72:0/3918238603 shutdown_connections 2023-12-12T09:55:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.135+0000 7f6b1fce7640 1 -- 172.21.15.72:0/3918238603 wait complete. 2023-12-12T09:55:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.137+0000 7f6b1fce7640 1 Processor -- start 2023-12-12T09:55:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.137+0000 7f6b1fce7640 1 -- start start 2023-12-12T09:55:36.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1fce7640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b1806de20 0x7f6b1814aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1fce7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b18157a60 0x7f6b1815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1fce7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b18155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1fce7640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1814b420 con 0x7f6b1806de20 2023-12-12T09:55:36.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1fce7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1815c010 con 0x7f6b18157a60 2023-12-12T09:55:36.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1fce7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1815c2f0 con 0x7f6b1815e2b0 2023-12-12T09:55:36.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1da83640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b1806de20 0x7f6b1814aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1da83640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b1806de20 0x7f6b1814aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37584/0 (socket says 172.21.15.72:37584) 2023-12-12T09:55:36.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1da83640 1 -- 172.21.15.72:0/1648396931 learned_addr learned my addr 172.21.15.72:0/1648396931 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:36.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.138+0000 7f6b1e284640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b18155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1d282640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b18157a60 0x7f6b1815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1da83640 1 -- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 msgr2=0x7f6b18155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1da83640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b18155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1da83640 1 -- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b18157a60 msgr2=0x7f6b1815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1da83640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b18157a60 0x7f6b1815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1da83640 1 -- 172.21.15.72:0/1648396931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10009b30 con 0x7f6b1806de20 2023-12-12T09:55:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1d282640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b18157a60 0x7f6b1815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:55:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1da83640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b1806de20 0x7f6b1814aee0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e9b0 tx=0x7f6b1400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b0e7fc640 1 -- 172.21.15.72:0/1648396931 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14009670 con 0x7f6b1806de20 2023-12-12T09:55:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b0e7fc640 1 -- 172.21.15.72:0/1648396931 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1400c9a0 con 0x7f6b1806de20 2023-12-12T09:55:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b0e7fc640 1 -- 172.21.15.72:0/1648396931 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400cd50 con 0x7f6b1806de20 2023-12-12T09:55:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.139+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1815c5d0 con 0x7f6b1806de20 2023-12-12T09:55:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.140+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b18155330 con 0x7f6b1806de20 2023-12-12T09:55:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.141+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b180745d0 con 0x7f6b1806de20 2023-12-12T09:55:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.141+0000 7f6b0e7fc640 1 -- 172.21.15.72:0/1648396931 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b14013470 con 0x7f6b1806de20 2023-12-12T09:55:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.142+0000 7f6b0e7fc640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6afc06ae40 0x7f6afc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.142+0000 7f6b0e7fc640 1 -- 172.21.15.72:0/1648396931 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f6b14089740 con 0x7f6b1806de20 2023-12-12T09:55:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.143+0000 7f6b1d282640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6afc06ae40 0x7f6afc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.143+0000 7f6b1d282640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6afc06ae40 0x7f6afc06d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b180ce210 tx=0x7f6b08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:36.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.147+0000 7f6b0e7fc640 1 -- 172.21.15.72:0/1648396931 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b14054910 con 0x7f6b1806de20 2023-12-12T09:55:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.331+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 --> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b1806e2b0 con 0x7f6afc06ae40 2023-12-12T09:55:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.334+0000 7f6b0e7fc640 1 -- 172.21.15.72:0/1648396931 <== mgr.4107 v2:172.21.15.72:6832/2070655694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48260 (secure 0 0 0) 0x7f6b080013e0 con 0x7f6afc06ae40 2023-12-12T09:55:36.336 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:55:36.337 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-12-12T09:55:36.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.336+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6afc06ae40 msgr2=0x7f6afc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.336+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6afc06ae40 0x7f6afc06d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b180ce210 tx=0x7f6b08009040 comp rx=0 tx=0).stop 2023-12-12T09:55:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b1806de20 msgr2=0x7f6b1814aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b1806de20 0x7f6b1814aee0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e9b0 tx=0x7f6b1400ee70 comp rx=0 tx=0).stop 2023-12-12T09:55:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 shutdown_connections 2023-12-12T09:55:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f6afc06ae40 0x7f6afc06d300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b1815e2b0 0x7f6b18155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b18157a60 0x7f6b1815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 --2- 172.21.15.72:0/1648396931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b1806de20 0x7f6b1814aee0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 >> 172.21.15.72:0/1648396931 conn(0x7f6b180b0a90 msgr2=0x7f6b18070180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 shutdown_connections 2023-12-12T09:55:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.337+0000 7f6b1fce7640 1 -- 172.21.15.72:0/1648396931 wait complete. 2023-12-12T09:55:36.356 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-12-12T09:55:35.609735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255872,"kb_used_data":6428,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718848,"statfs":{"total":773094113280,"available":772832100352,"internally_reserved":0,"allocated":6582272,"data_stored":3571719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003279"},"pg_stats":[{"pgid":"1.0","version":"56'119","reported_seq":208,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-12-12T09:55:15.973264+0000","last_change":"2023-12-12T09:55:15.973329+0000","last_active":"2023-12-12T09:55:15.973264+0000","last_peered":"2023-12-12T09:55:15.973264+0000","last_clean":"2023-12-12T09:55:15.973264+0000","last_became_active":"2023-12-12T09:52:03.157960+0000","last_became_peered":"2023-12-12T09:52:03.157960+0000","last_unstale":"2023-12-12T09:55:15.973264+0000","last_undegraded":"2023-12-12T09:55:15.973264+0000","last_fullsized":"2023-12-12T09:55:15.973264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'119","last_scrub_stamp":"2023-12-12T09:55:15.973104+0000","last_deep_scrub":"56'119","last_deep_scrub_stamp":"2023-12-12T09:55:15.973104+0000","last_clean_scrub_stamp":"2023-12-12T09:55:15.973104+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:02:01.079279+0000","scrub_duration":0.0099172649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1269760,"data_stored":889623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0880000000000001,"15min":1.0900000000000001},"min":{"1min":0.61899999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6799999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1559999999999999,"15min":1.1419999999999999},"min":{"1min":1.0089999999999999,"5min":0.622,"15min":0.622},"max":{"1min":1.6950000000000001,"5min":1.855,"15min":1.855},"last":1.2390000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0369999999999999,"15min":1.024},"min":{"1min":0.75800000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.361},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.135,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5189999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.294}]},{"osd":2,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3280000000000001,"15min":1.3819999999999999},"min":{"1min":0.83499999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.8420000000000001,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.369,"15min":1.448},"min":{"1min":0.877,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5589999999999999,"5min":6.6210000000000004,"15min":6.6210000000000004},"last":1.1990000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3520000000000001,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8899999999999999,"5min":2.23,"15min":2.23},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.4079999999999999,"15min":1.4179999999999999},"min":{"1min":0.91800000000000004,"5min":0.746,"15min":0.746},"max":{"1min":1.845,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.331}]},{"osd":4,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3939999999999999,"15min":1.4299999999999999},"min":{"1min":0.78600000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.76,"5min":2.649,"15min":2.649},"last":1.716},{"interface":"front","average":{"1min":1.329,"5min":1.3859999999999999,"15min":1.421},"min":{"1min":0.89600000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7749999999999999,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.0049999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.498,"15min":1.556},"min":{"1min":0.93999999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.766,"5min":2.5979999999999999,"15min":2.5979999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.5449999999999999,"15min":1.5620000000000001},"min":{"1min":1.0009999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.901,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.645}]},{"osd":6,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.5880000000000001,"15min":1.6180000000000001},"min":{"1min":1.1850000000000001,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.921,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.522},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.571,"15min":1.617},"min":{"1min":1.282,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.736,"5min":2.8300000000000001,"15min":2.8300000000000001},"last":1.5609999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0529999999999999,"15min":1.0449999999999999},"min":{"1min":0.76000000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5820000000000001,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.115,"15min":1.107},"min":{"1min":0.754,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.702,"5min":1.865,"15min":1.865},"last":0.83099999999999996}]},{"osd":1,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2330000000000001,"15min":1.2270000000000001},"min":{"1min":0.81999999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7250000000000001,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.95699999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.5940000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.2350000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.139,"15min":1.133},"min":{"1min":0.747,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.393,"5min":1.859,"15min":1.859},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2769999999999999,"15min":1.2889999999999999},"min":{"1min":0.754,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.5589999999999999,"5min":2.028,"15min":2.028},"last":0.90700000000000003}]},{"osd":3,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3200000000000001,"15min":1.3109999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.071},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.26,"15min":1.256},"min":{"1min":0.90900000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7210000000000001,"5min":1.994,"15min":1.994},"last":1.383}]},{"osd":4,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.379,"15min":1.393},"min":{"1min":0.92000000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6799999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.5189999999999999,"15min":1.5369999999999999},"min":{"1min":1.1359999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.1269999999999998,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":0.96199999999999997}]},{"osd":5,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3819999999999999,"15min":1.377},"min":{"1min":1.0229999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.6679999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.996,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":1.7829999999999999,"5min":3.1989999999999998,"15min":3.1989999999999998},"last":1.3640000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:55:01 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.6619999999999999,"15min":1.696},"min":{"1min":0.93999999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2269999999999999,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":1.151},{"interface":"front","average":{"1min":1.504,"5min":1.5189999999999999,"15min":1.5129999999999999},"min":{"1min":1.288,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8200000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.4970000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.091,"15min":1.042},"min":{"1min":0.89800000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.823,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.409},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.194,"15min":1.24},"min":{"1min":0.68000000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.734,"5min":3.694,"15min":3.694},"last":1}]},{"osd":1,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.2689999999999999,"15min":1.331},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7230000000000001,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0820000000000001,"15min":1.0940000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.5069999999999999,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.458}]},{"osd":2,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1240000000000001,"15min":1.087},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.145,"15min":1.1040000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7470000000000001,"5min":1.895,"15min":1.895},"last":1.526}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.351,"15min":1.367},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8620000000000001,"5min":3.944,"15min":3.944},"last":1.486},{"interface":"front","average":{"1min":1.377,"5min":1.3149999999999999,"15min":1.2989999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.556}]},{"osd":4,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3460000000000001,"15min":1.337},"min":{"1min":0.80300000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.962,"5min":2.536,"15min":2.536},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4910000000000001,"15min":1.5409999999999999},"min":{"1min":0.82999999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.218,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.341}]},{"osd":6,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.446,"15min":1.4590000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.1669999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.494,"15min":1.496},"min":{"1min":0.94799999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.0990000000000002,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.621}]},{"osd":7,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.5760000000000001,"15min":1.6180000000000001},"min":{"1min":0.70699999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1339999999999999,"5min":4.1500000000000004,"15min":4.1500000000000004},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.6140000000000001,"15min":1.653},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.4350000000000001,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.222}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":298236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.101,"15min":1.079},"min":{"1min":0.626,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.639,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.224,"15min":1.1259999999999999},"min":{"1min":0.745,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7410000000000001,"5min":7.29,"15min":7.29},"last":1.6080000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2529999999999999,"15min":1.179},"min":{"1min":0.91700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.587,"5min":7.3360000000000003,"15min":7.3360000000000003},"last":1.151},{"interface":"front","average":{"1min":1.22,"5min":1.282,"15min":1.284},"min":{"1min":0.88900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.595,"5min":6.7530000000000001,"15min":6.7530000000000001},"last":1.194}]},{"osd":2,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.165,"15min":1.1539999999999999},"min":{"1min":0.69899999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6339999999999999,"5min":4.891,"15min":4.891},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.375,"15min":1.3120000000000001},"min":{"1min":0.747,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8169999999999999,"5min":7.4100000000000001,"15min":7.4100000000000001},"last":1.732}]},{"osd":3,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3200000000000001,"15min":1.306},"min":{"1min":0.81100000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.635,"5min":6.6200000000000001,"15min":6.6200000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.323,"15min":1.3420000000000001},"min":{"1min":0.74099999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0720000000000001,"5min":4.7409999999999997,"15min":4.7409999999999997},"last":1.262}]},{"osd":5,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.4970000000000001,"15min":1.427},"min":{"1min":0.81399999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8899999999999999,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":1.391},{"interface":"front","average":{"1min":1.409,"5min":1.4930000000000001,"15min":1.4850000000000001},"min":{"1min":0.89600000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.8959999999999999,"5min":6.8499999999999996,"15min":6.8499999999999996},"last":1.425}]},{"osd":6,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.405,"15min":1.4039999999999999},"min":{"1min":0.86799999999999999,"5min":0.629,"15min":0.629},"max":{"1min":2.1880000000000002,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.514},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.6080000000000001,"15min":1.5580000000000001},"min":{"1min":0.95299999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9730000000000001,"5min":7.3739999999999997,"15min":7.3739999999999997},"last":1.8029999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.587,"15min":1.569},"min":{"1min":1.046,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.0099999999999998,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.696},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.6100000000000001,"15min":1.577},"min":{"1min":1.137,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.23,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.655}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82099999999999995,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4510000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93000000000000005,"15min":0.93999999999999995},"min":{"1min":0.629,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.206,"5min":1.577,"15min":1.577},"last":0.78600000000000003}]},{"osd":1,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.873,"15min":0.871},"min":{"1min":0.55600000000000005,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1499999999999999,"5min":1.548,"15min":1.548},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.105,"5min":0.996,"15min":0.98199999999999998},"min":{"1min":0.66700000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.1830000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.048,"15min":1.0629999999999999},"min":{"1min":0.67500000000000004,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4710000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.52},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0189999999999999,"15min":1.0129999999999999},"min":{"1min":0.504,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.347}]},{"osd":4,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.81599999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2360000000000002,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.254},{"interface":"front","average":{"1min":1.367,"5min":1.272,"15min":1.268},"min":{"1min":1.046,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.782,"5min":1.964,"15min":1.964},"last":1.2090000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3009999999999999,"15min":1.3169999999999999},"min":{"1min":0.97899999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7290000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.327,"15min":1.323},"min":{"1min":0.83699999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.748,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.29}]},{"osd":6,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.321,"15min":1.329},"min":{"1min":0.68200000000000005,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.919,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.327,"15min":1.3300000000000001},"min":{"1min":0.99399999999999999,"5min":0.627,"15min":0.627},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.3240000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:55:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3580000000000001,"15min":1.3560000000000001},"min":{"1min":1.081,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8080000000000001,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.593},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.022,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.831,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.4510000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":299255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.33500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4510000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.871,"15min":0.90300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.508,"5min":1.752,"15min":1.752},"last":0.75}]},{"osd":1,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.90800000000000003,"15min":0.92400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.573,"5min":1.603,"15min":1.603},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.96499999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4830000000000001,"5min":1.633,"15min":1.633},"last":0.99299999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90100000000000002,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.494,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.202,"5min":1.157,"15min":1.1839999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.754,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.4950000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.296,"15min":1.321},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.6799999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.339},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2390000000000001,"15min":1.2549999999999999},"min":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.5900000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.2929999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.177,"15min":1.208},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.573,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3,"15min":1.3200000000000001},"min":{"1min":0.72999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6359999999999999,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.4330000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.337,"15min":1.381},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.206},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.345,"15min":1.387},"min":{"1min":0.872,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6850000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.2529999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:55:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.4159999999999999,"15min":1.468},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.742,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.377,"15min":1.4219999999999999},"min":{"1min":0.878,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.79,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":1.1830000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":298236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.752},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6719999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.3100000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.88300000000000001,"15min":0.876},"min":{"1min":0.47599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.337,"5min":1.712,"15min":1.712},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.151,"5min":1.024,"15min":1.004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.1220000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0269999999999999,"15min":1.0389999999999999},"min":{"1min":0.61699999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.655,"5min":1.665,"15min":1.665},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.532,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1659999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1160000000000001,"15min":1.1299999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6000000000000001,"5min":2.056,"15min":2.056},"last":1.212},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.3069999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.885,"5min":2.427,"15min":2.427},"last":1.2649999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3520000000000001,"15min":1.3620000000000001},"min":{"1min":0.85399999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8149999999999999,"5min":3.476,"15min":3.476},"last":1.385},{"interface":"front","average":{"1min":1.359,"5min":1.3300000000000001,"15min":1.3500000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.744,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.3540000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.405,"15min":1.425},"min":{"1min":0.89600000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9330000000000001,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.341,"15min":1.363},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.4510000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3839999999999999,"15min":1.3979999999999999},"min":{"1min":0.93799999999999994,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9950000000000001,"5min":3.052,"15min":3.052},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.427,"15min":1.4470000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8360000000000001,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":2.0110000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1265664,"data_stored":888604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.82699999999999996,"15min":0.83899999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.526,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.92500000000000004,"15min":0.99299999999999999},"min":{"1min":0.41499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.1000000000000001,"5min":4.0819999999999999,"15min":4.0819999999999999},"last":1.087}]},{"osd":2,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.96599999999999997,"15min":0.98899999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.5309999999999999,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0329999999999999,"15min":1.077},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.734,"5min":3.9940000000000002,"15min":3.9940000000000002},"last":1.2569999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.97899999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.556,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1359999999999999},"min":{"1min":0.495,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5609999999999999,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.325}]},{"osd":4,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.1850000000000001,"15min":1.1910000000000001},"min":{"1min":0.71799999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.7789999999999999,"5min":2.738,"15min":2.738},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.302,"15min":1.347},"min":{"1min":0.69799999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6180000000000001,"5min":3.875,"15min":3.875},"last":1.694}]},{"osd":5,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2430000000000001,"15min":1.268},"min":{"1min":0.76400000000000001,"5min":0.755,"15min":0.755},"max":{"1min":1.6000000000000001,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.393},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2629999999999999,"15min":1.2869999999999999},"min":{"1min":0.66200000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.677,"5min":2.54,"15min":2.54},"last":1.605}]},{"osd":6,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.278,"15min":1.2789999999999999},"min":{"1min":0.753,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.732,"5min":2.117,"15min":2.117},"last":1.522},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.383,"15min":1.429},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.744,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":1.4830000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.359,"15min":1.379},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.806,"5min":2.931,"15min":2.931},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.448,"15min":1.53},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.851,"5min":4.548,"15min":4.548},"last":1.8200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:55:36.358 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-12T09:55:36.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.561+0000 7ff946d16640 1 Processor -- start 2023-12-12T09:55:36.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.561+0000 7ff946d16640 1 -- start start 2023-12-12T09:55:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.562+0000 7ff946d16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff940142de0 0x7ff9400cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff946d16640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff9401432b0 0x7ff94006d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff946d16640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9400b4980 con 0x7ff94006da40 2023-12-12T09:55:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff946d16640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94006ded0 con 0x7ff9401432b0 2023-12-12T09:55:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff946d16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400ce2a0 con 0x7ff940142de0 2023-12-12T09:55:36.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff93ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff9401432b0 0x7ff94006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff93ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff9401432b0 0x7ff94006d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37594/0 (socket says 172.21.15.72:37594) 2023-12-12T09:55:36.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff93ffff640 1 -- 172.21.15.72:0/1133114932 learned_addr learned my addr 172.21.15.72:0/1133114932 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:36.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff944ab2640 1 --2- 172.21.15.72:0/1133114932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff940142de0 0x7ff9400cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff93ffff640 1 -- 172.21.15.72:0/1133114932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff940142de0 msgr2=0x7ff9400cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:36.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff93ffff640 1 --2- 172.21.15.72:0/1133114932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff940142de0 0x7ff9400cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.563+0000 7ff93ffff640 1 -- 172.21.15.72:0/1133114932 >> v1:172.21.15.113:6789/0 conn(0x7ff94006da40 legacy=0x7ff94015dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:36.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.564+0000 7ff93ffff640 1 -- 172.21.15.72:0/1133114932 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940071260 con 0x7ff9401432b0 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.565+0000 7ff93ffff640 1 --2- 172.21.15.72:0/1133114932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9401432b0 0x7ff94006d500 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009870 tx=0x7ff92c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f553f000ded119 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.565+0000 7ff93f7fe640 1 -- 172.21.15.72:0/1133114932 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0317c0 con 0x7ff9401432b0 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.565+0000 7ff93f7fe640 1 -- 172.21.15.72:0/1133114932 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c031920 con 0x7ff9401432b0 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.565+0000 7ff93f7fe640 1 -- 172.21.15.72:0/1133114932 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c031cb0 con 0x7ff9401432b0 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.565+0000 7ff946d16640 1 -- 172.21.15.72:0/1133114932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9401432b0 msgr2=0x7ff94006d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.565+0000 7ff946d16640 1 --2- 172.21.15.72:0/1133114932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9401432b0 0x7ff94006d500 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009870 tx=0x7ff92c02b860 comp rx=0 tx=0).stop 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.566+0000 7ff946d16640 1 -- 172.21.15.72:0/1133114932 shutdown_connections 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.566+0000 7ff946d16640 1 --2- 172.21.15.72:0/1133114932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9401432b0 0x7ff94006d500 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.566+0000 7ff946d16640 1 --2- 172.21.15.72:0/1133114932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff940142de0 0x7ff9400cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.566+0000 7ff946d16640 1 -- 172.21.15.72:0/1133114932 >> 172.21.15.72:0/1133114932 conn(0x7ff9400cb850 msgr2=0x7ff9400cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:36.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.566+0000 7ff946d16640 1 -- 172.21.15.72:0/1133114932 shutdown_connections 2023-12-12T09:55:36.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.566+0000 7ff946d16640 1 -- 172.21.15.72:0/1133114932 wait complete. 2023-12-12T09:55:36.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.567+0000 7ff946d16640 1 Processor -- start 2023-12-12T09:55:36.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.568+0000 7ff946d16640 1 -- start start 2023-12-12T09:55:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.568+0000 7ff946d16640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff94006da40 0x7ff940159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.568+0000 7ff946d16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff940142de0 0x7ff94013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.568+0000 7ff946d16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9401432b0 0x7ff940071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.568+0000 7ff946d16640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff940139170 con 0x7ff94006da40 2023-12-12T09:55:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.568+0000 7ff946d16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94013b9d0 con 0x7ff940142de0 2023-12-12T09:55:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.568+0000 7ff946d16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff940139410 con 0x7ff9401432b0 2023-12-12T09:55:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.569+0000 7ff944ab2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff94006da40 0x7ff940159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.569+0000 7ff944ab2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff94006da40 0x7ff940159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37596/0 (socket says 172.21.15.72:37596) 2023-12-12T09:55:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.569+0000 7ff944ab2640 1 -- 172.21.15.72:0/3793857406 learned_addr learned my addr 172.21.15.72:0/3793857406 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:36.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.569+0000 7ff944ab2640 1 -- 172.21.15.72:0/3793857406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9401432b0 msgr2=0x7ff940071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.569+0000 7ff944ab2640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9401432b0 0x7ff940071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.569+0000 7ff944ab2640 1 -- 172.21.15.72:0/3793857406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff940142de0 msgr2=0x7ff94013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.569+0000 7ff944ab2640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff940142de0 0x7ff94013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.569+0000 7ff944ab2640 1 -- 172.21.15.72:0/3793857406 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0094d0 con 0x7ff94006da40 2023-12-12T09:55:36.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.570+0000 7ff944ab2640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff94006da40 0x7ff940159420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff93400d800 tx=0x7ff93400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:36.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.570+0000 7ff93d7fa640 1 -- 172.21.15.72:0/3793857406 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93400b6c0 con 0x7ff94006da40 2023-12-12T09:55:36.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.570+0000 7ff93d7fa640 1 -- 172.21.15.72:0/3793857406 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93400b820 con 0x7ff94006da40 2023-12-12T09:55:36.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.570+0000 7ff93d7fa640 1 -- 172.21.15.72:0/3793857406 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93400bbb0 con 0x7ff94006da40 2023-12-12T09:55:36.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.570+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9401396f0 con 0x7ff94006da40 2023-12-12T09:55:36.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.570+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94013bcb0 con 0x7ff94006da40 2023-12-12T09:55:36.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.571+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9400745d0 con 0x7ff94006da40 2023-12-12T09:55:36.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.572+0000 7ff93d7fa640 1 -- 172.21.15.72:0/3793857406 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff93400bd10 con 0x7ff94006da40 2023-12-12T09:55:36.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.573+0000 7ff93d7fa640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff92406ae40 0x7ff92406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:36.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.573+0000 7ff93d7fa640 1 -- 172.21.15.72:0/3793857406 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff934089a00 con 0x7ff94006da40 2023-12-12T09:55:36.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.573+0000 7ff93ffff640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff92406ae40 0x7ff92406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:36.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.574+0000 7ff93ffff640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff92406ae40 0x7ff92406d300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff92c036000 tx=0x7ff92c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:36.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.577+0000 7ff93d7fa640 1 -- 172.21.15.72:0/3793857406 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff934054c60 con 0x7ff94006da40 2023-12-12T09:55:36.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.769+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7ff9400cc650 con 0x7ff94006da40 2023-12-12T09:55:36.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.775+0000 7ff93d7fa640 1 -- 172.21.15.72:0/3793857406 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff934058280 con 0x7ff94006da40 2023-12-12T09:55:36.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.778+0000 7ff93d7fa640 1 -- 172.21.15.72:0/3793857406 <== mon.0 v2:172.21.15.72:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff934054c60 con 0x7ff94006da40 2023-12-12T09:55:36.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff92406ae40 msgr2=0x7ff92406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff92406ae40 0x7ff92406d300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff92c036000 tx=0x7ff92c030000 comp rx=0 tx=0).stop 2023-12-12T09:55:36.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff94006da40 msgr2=0x7ff940159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:36.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff94006da40 0x7ff940159420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff93400d800 tx=0x7ff93400dcc0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 shutdown_connections 2023-12-12T09:55:36.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7ff92406ae40 0x7ff92406d300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9401432b0 0x7ff940071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff940142de0 0x7ff94013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 --2- 172.21.15.72:0/3793857406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff94006da40 0x7ff940159420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:36.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 >> 172.21.15.72:0/3793857406 conn(0x7ff9400cb850 msgr2=0x7ff94015d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:36.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.780+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 shutdown_connections 2023-12-12T09:55:36.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:36.781+0000 7ff946d16640 1 -- 172.21.15.72:0/3793857406 wait complete. 2023-12-12T09:55:36.805 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-12T09:55:36.806 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:55:37.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.014+0000 7f587c8b1640 1 Processor -- start 2023-12-12T09:55:37.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.014+0000 7f587c8b1640 1 -- start start 2023-12-12T09:55:37.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.015+0000 7f587c8b1640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f58780ae200 0x7f5878070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:37.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.015+0000 7f587c8b1640 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5878071f90 con 0x7f58780abd20 2023-12-12T09:55:37.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f587c8b1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5878072270 con 0x7f587815e5d0 2023-12-12T09:55:37.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f587c8b1640 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5878070070 con 0x7f58780ae200 2023-12-12T09:55:37.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f586ffff640 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f58780ae200 0x7f5878070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:37.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f587759e640 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f58780abd20 0x7f58780cded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:34522/0 (socket says 172.21.15.72:34522) 2023-12-12T09:55:37.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f587759e640 1 -- 172.21.15.72:0/1065115914 learned_addr learned my addr 172.21.15.72:0/1065115914 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-12-12T09:55:37.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f586ffff640 1 -- 172.21.15.72:0/1065115914 >> v1:172.21.15.113:6789/0 conn(0x7f587815e5d0 legacy=0x7f587806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:55:37.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f586ffff640 1 -- 172.21.15.72:0/1065115914 >> v1:172.21.15.72:6789/0 conn(0x7f58780abd20 legacy=0x7f58780cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:37.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f587659c640 1 -- 172.21.15.72:0/1065115914 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282261656 0 0) 0x7f5878071f90 con 0x7f58780abd20 2023-12-12T09:55:37.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.016+0000 7f586ffff640 1 -- 172.21.15.72:0/1065115914 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878070350 con 0x7f58780ae200 2023-12-12T09:55:37.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.017+0000 7f586ffff640 1 --2- 172.21.15.72:0/1065115914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780ae200 0x7f5878070b50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f58600097a0 tx=0x7f586002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec3894bd3caf88c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587659c640 1 -- 172.21.15.72:0/1065115914 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586002e070 con 0x7f58780ae200 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587659c640 1 -- 172.21.15.72:0/1065115914 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f58600315a0 con 0x7f58780ae200 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587659c640 1 -- 172.21.15.72:0/1065115914 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860031950 con 0x7f58780ae200 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587c8b1640 1 -- 172.21.15.72:0/1065115914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780ae200 msgr2=0x7f5878070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587c8b1640 1 --2- 172.21.15.72:0/1065115914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780ae200 0x7f5878070b50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f58600097a0 tx=0x7f586002b840 comp rx=0 tx=0).stop 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587c8b1640 1 -- 172.21.15.72:0/1065115914 shutdown_connections 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587c8b1640 1 --2- 172.21.15.72:0/1065115914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780ae200 0x7f5878070b50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587c8b1640 1 -- 172.21.15.72:0/1065115914 >> 172.21.15.72:0/1065115914 conn(0x7f58780b0860 msgr2=0x7f58780b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:37.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.018+0000 7f587c8b1640 1 -- 172.21.15.72:0/1065115914 shutdown_connections 2023-12-12T09:55:37.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.019+0000 7f587c8b1640 1 -- 172.21.15.72:0/1065115914 wait complete. 2023-12-12T09:55:37.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.020+0000 7f587c8b1640 1 Processor -- start 2023-12-12T09:55:37.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.020+0000 7f587c8b1640 1 -- start start 2023-12-12T09:55:37.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.021+0000 7f587c8b1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780abd20 0x7f587814c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:37.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.021+0000 7f587c8b1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58780ae200 0x7f5878151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:37.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.021+0000 7f587c8b1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f587815e5d0 0x7f5878154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:37.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.021+0000 7f587c8b1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58780b2060 con 0x7f58780abd20 2023-12-12T09:55:37.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.021+0000 7f587c8b1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5878155470 con 0x7f58780ae200 2023-12-12T09:55:37.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.021+0000 7f587c8b1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58780b22b0 con 0x7f587815e5d0 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.021+0000 7f5876d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780abd20 0x7f587814c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.022+0000 7f5876d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780abd20 0x7f587814c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37616/0 (socket says 172.21.15.72:37616) 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.022+0000 7f5876d9d640 1 -- 172.21.15.72:0/1754438492 learned_addr learned my addr 172.21.15.72:0/1754438492 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.022+0000 7f5876d9d640 1 -- 172.21.15.72:0/1754438492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f587815e5d0 msgr2=0x7f5878154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.022+0000 7f5876d9d640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f587815e5d0 0x7f5878154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.022+0000 7f5876d9d640 1 -- 172.21.15.72:0/1754438492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58780ae200 msgr2=0x7f5878151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.022+0000 7f5876d9d640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58780ae200 0x7f5878151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.022+0000 7f5876d9d640 1 -- 172.21.15.72:0/1754438492 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860009400 con 0x7f58780abd20 2023-12-12T09:55:37.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.022+0000 7f5876d9d640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780abd20 0x7f587814c9d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f586800b3f0 tx=0x7f586800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:37.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.023+0000 7f586f7fe640 1 -- 172.21.15.72:0/1754438492 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868011830 con 0x7f58780abd20 2023-12-12T09:55:37.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.023+0000 7f586f7fe640 1 -- 172.21.15.72:0/1754438492 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5868011990 con 0x7f58780abd20 2023-12-12T09:55:37.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.023+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587806c680 con 0x7f58780abd20 2023-12-12T09:55:37.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.023+0000 7f586f7fe640 1 -- 172.21.15.72:0/1754438492 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868011d60 con 0x7f58780abd20 2023-12-12T09:55:37.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.023+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58780b2870 con 0x7f58780abd20 2023-12-12T09:55:37.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.024+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5878074450 con 0x7f58780abd20 2023-12-12T09:55:37.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.025+0000 7f586f7fe640 1 -- 172.21.15.72:0/1754438492 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f586801a400 con 0x7f58780abd20 2023-12-12T09:55:37.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.025+0000 7f586f7fe640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f585806ae70 0x7f585806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:37.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.026+0000 7f586f7fe640 1 -- 172.21.15.72:0/1754438492 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5868089440 con 0x7f58780abd20 2023-12-12T09:55:37.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.026+0000 7f586ffff640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f585806ae70 0x7f585806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:37.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.026+0000 7f586ffff640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f585806ae70 0x7f585806d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5860036000 tx=0x7f5860030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:37.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.030+0000 7f586f7fe640 1 -- 172.21.15.72:0/1754438492 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5868054580 con 0x7f58780abd20 2023-12-12T09:55:37.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.255+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58780aca10 con 0x7f58780abd20 2023-12-12T09:55:37.258 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.256+0000 7f586f7fe640 1 -- 172.21.15.72:0/1754438492 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f5868057ba0 con 0x7f58780abd20 2023-12-12T09:55:37.258 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-12-12T09:55:37.258 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/130800926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":130800926},{"type":"v1","addr":"172.21.15.72:6836","nonce":130800926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4931,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/457868261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":457868261},{"type":"v1","addr":"172.21.15.113:6834","nonce":457868261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/318582100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":318582100},{"type":"v1","addr":"172.21.15.72:6837","nonce":318582100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2380516950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6833","nonce":2380516950},{"type":"v1","addr":"172.21.15.113:6835","nonce":2380516950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-12-12T09:55:37.259 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2023-12-12T09:55:37.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f585806ae70 msgr2=0x7f585806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:37.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f585806ae70 0x7f585806d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5860036000 tx=0x7f5860030000 comp rx=0 tx=0).stop 2023-12-12T09:55:37.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780abd20 msgr2=0x7f587814c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:37.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780abd20 0x7f587814c9d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f586800b3f0 tx=0x7f586800b8b0 comp rx=0 tx=0).stop 2023-12-12T09:55:37.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 shutdown_connections 2023-12-12T09:55:37.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:6832/2070655694,v1:172.21.15.72:6833/2070655694] conn(0x7f585806ae70 0x7f585806d330 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:37.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f587815e5d0 0x7f5878154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:37.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58780ae200 0x7f5878151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:37.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 --2- 172.21.15.72:0/1754438492 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58780abd20 0x7f587814c9d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:37.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.258+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 >> 172.21.15.72:0/1754438492 conn(0x7f58780b0860 msgr2=0x7f587806c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:37.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.259+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 shutdown_connections 2023-12-12T09:55:37.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-12-12T09:55:37.259+0000 7f587c8b1640 1 -- 172.21.15.72:0/1754438492 wait complete. 2023-12-12T09:55:37.279 DEBUG:tasks.cephfs.filesystem:already dead... 2023-12-12T09:55:37.279 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-12T09:55:37.280 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-12T09:55:37.280 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.307 INFO:tasks.ceph.mds.a:Stopped 2023-12-12T09:55:37.307 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-12T09:55:37.307 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.368 INFO:tasks.ceph.mds.c:Stopped 2023-12-12T09:55:37.369 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-12T09:55:37.369 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.393 INFO:tasks.ceph.mds.b:Stopped 2023-12-12T09:55:37.394 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-12T09:55:37.394 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.460 INFO:tasks.ceph.mds.d:Stopped 2023-12-12T09:55:37.460 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-12T09:55:37.460 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-12T09:55:37.460 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.506 INFO:tasks.ceph.osd.0:Stopped 2023-12-12T09:55:37.507 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-12T09:55:37.507 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.576 INFO:tasks.ceph.osd.1:Stopped 2023-12-12T09:55:37.576 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-12T09:55:37.577 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.637 INFO:tasks.ceph.osd.2:Stopped 2023-12-12T09:55:37.638 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-12T09:55:37.638 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.705 INFO:tasks.ceph.osd.3:Stopped 2023-12-12T09:55:37.705 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-12T09:55:37.705 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.764 INFO:tasks.ceph.osd.4:Stopped 2023-12-12T09:55:37.765 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-12T09:55:37.765 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.831 INFO:tasks.ceph.osd.5:Stopped 2023-12-12T09:55:37.832 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-12T09:55:37.832 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.898 INFO:tasks.ceph.osd.6:Stopped 2023-12-12T09:55:37.899 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-12T09:55:37.899 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:37.982 INFO:tasks.ceph.osd.7:Stopped 2023-12-12T09:55:37.982 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-12T09:55:37.983 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-12T09:55:37.983 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:38.062 INFO:tasks.ceph.mgr.y:Stopped 2023-12-12T09:55:38.063 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-12T09:55:38.063 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:38.154 INFO:tasks.ceph.mgr.x:Stopped 2023-12-12T09:55:38.154 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-12T09:55:38.155 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-12T09:55:38.155 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:38.199 INFO:tasks.ceph.mon.a:Stopped 2023-12-12T09:55:38.199 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-12T09:55:38.199 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:38.242 INFO:tasks.ceph.mon.b:Stopped 2023-12-12T09:55:38.242 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-12T09:55:38.243 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:55:38.279 INFO:tasks.ceph.mon.c:Stopped 2023-12-12T09:55:38.280 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-12T09:55:38.280 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-12T09:55:38.386 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:55:38.387 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-12T09:55:38.670 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:55:38.670 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-12T09:55:38.823 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:55:38.823 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-12T09:55:38.965 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:55:38.965 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-12T09:55:39.128 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:55:39.128 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-12T09:55:39.403 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:55:39.403 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-12T09:55:39.563 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:55:39.563 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-12T09:55:39.708 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:55:39.709 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-12T09:55:39.867 INFO:tasks.ceph:Archiving mon data... 2023-12-12T09:55:39.869 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/data/mon.a.tgz 2023-12-12T09:55:39.869 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2023-12-12T09:55:39.929 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.ZV5jRkk5wq 2023-12-12T09:55:39.929 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ZV5jRkk5wq 2023-12-12T09:55:40.121 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.ZV5jRkk5wq 2023-12-12T09:55:40.278 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.ZV5jRkk5wq is 351KB 2023-12-12T09:55:40.391 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.ZV5jRkk5wq 2023-12-12T09:55:40.450 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/data/mon.b.tgz 2023-12-12T09:55:40.450 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2023-12-12T09:55:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.9v2eQe6qED 2023-12-12T09:55:40.512 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.9v2eQe6qED 2023-12-12T09:55:40.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.9v2eQe6qED 2023-12-12T09:55:40.895 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.9v2eQe6qED is 461KB 2023-12-12T09:55:41.006 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.9v2eQe6qED 2023-12-12T09:55:41.062 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/data/mon.c.tgz 2023-12-12T09:55:41.063 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2023-12-12T09:55:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.eLxJBdJynl 2023-12-12T09:55:41.108 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.eLxJBdJynl 2023-12-12T09:55:41.352 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.eLxJBdJynl 2023-12-12T09:55:41.503 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.eLxJBdJynl is 461KB 2023-12-12T09:55:41.613 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.eLxJBdJynl 2023-12-12T09:55:41.672 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-12T09:55:41.673 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-12T09:55:41.682 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-12T09:55:41.840 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2023-12-12T09:55:41.899 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2023-12-12T09:55:41.899 INFO:tasks.ceph:Archiving crash dumps... 2023-12-12T09:55:41.901 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/remote/smithi072/crash 2023-12-12T09:55:41.902 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-12T09:55:41.994 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/remote/smithi113/crash 2023-12-12T09:55:41.995 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-12T09:55:42.096 INFO:tasks.ceph:Shutting down logrotate 2023-12-12T09:55:42.096 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:55:42.105 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-12T09:55:42.230 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-12T09:55:42.329 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-12T09:55:42.430 INFO:tasks.ceph:Compressing logs... 2023-12-12T09:55:42.430 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-12T09:55:42.440 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-12T09:55:42.520 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94256.log 2023-12-12T09:55:42.521 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95516.log 2023-12-12T09:55:42.522 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94256.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-12-12T09:55:42.522 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94256.log.gz 2023-12-12T09:55:42.523 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107686.log 2023-12-12T09:55:42.523 INFO:teuthology.orchestra.run.smithi072.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95516.log.gz 2023-12-12T09:55:42.524 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97404.log 2023-12-12T09:55:42.526 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107686.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107686.log.gz 2023-12-12T09:55:42.526 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96815.log 2023-12-12T09:55:42.527 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97404.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97404.log.gz 2023-12-12T09:55:42.527 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99823.log 2023-12-12T09:55:42.529 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96815.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96815.log.gz 2023-12-12T09:55:42.529 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99276.log 2023-12-12T09:55:42.531 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99823.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99823.log.gz 2023-12-12T09:55:42.531 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101692.log 2023-12-12T09:55:42.532 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99276.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99276.log.gz 2023-12-12T09:55:42.532 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99529.log 2023-12-12T09:55:42.533 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101692.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101692.log.gz 2023-12-12T09:55:42.533 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106752.log 2023-12-12T09:55:42.534 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-12-12T09:55:42.536 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99529.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99529.log.gz 2023-12-12T09:55:42.536 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96228.log 2023-12-12T09:55:42.537 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106752.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106752.log.gz 2023-12-12T09:55:42.537 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94554.log 2023-12-12T09:55:42.542 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96228.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96228.log.gz 2023-12-12T09:55:42.542 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108734.log 2023-12-12T09:55:42.542 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94554.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94554.log.gz 2023-12-12T09:55:42.542 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103167.log 2023-12-12T09:55:42.542 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108734.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108734.log.gz 2023-12-12T09:55:42.542 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104628.log 2023-12-12T09:55:42.542 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103167.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103167.log.gz 2023-12-12T09:55:42.543 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102490.log 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88689.log 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88849.log 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88929.log 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88689.log.gz 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88849.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88849.log.gz 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88929.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.88929.log.gz /var/log/ceph/ceph-mds.b.log 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr: 2023-12-12T09:55:42.544 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88809.log 2023-12-12T09:55:42.545 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-12-12T09:55:42.545 INFO:teuthology.orchestra.run.smithi113.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-12T09:55:42.547 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88809.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88809.log.gz 2023-12-12T09:55:42.547 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.94012.log 2023-12-12T09:55:42.547 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88729.log 2023-12-12T09:55:42.547 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.3.94012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.94012.log.gz 2023-12-12T09:55:42.548 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104628.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104628.log.gz 2023-12-12T09:55:42.548 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108398.log 2023-12-12T09:55:42.548 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102490.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102490.log.gz 2023-12-12T09:55:42.548 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94162.log 2023-12-12T09:55:42.549 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108398.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108398.log.gz 2023-12-12T09:55:42.549 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95658.log 2023-12-12T09:55:42.549 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94162.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94162.log.gz 2023-12-12T09:55:42.549 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99613.log 2023-12-12T09:55:42.549 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95658.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95658.log.gz 2023-12-12T09:55:42.549 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106624.log 2023-12-12T09:55:42.551 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-12-12T09:55:42.551 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88729.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-12-12T09:55:42.551 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88729.log.gz 2023-12-12T09:55:42.552 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88889.log 2023-12-12T09:55:42.552 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99613.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99613.log.gz 2023-12-12T09:55:42.552 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103377.log 2023-12-12T09:55:42.552 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106624.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106624.log.gz 2023-12-12T09:55:42.552 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102827.log 2023-12-12T09:55:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88649.log 2023-12-12T09:55:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88889.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88889.log.gz 2023-12-12T09:55:42.556 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103377.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103377.log.gz 2023-12-12T09:55:42.556 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104922.log 2023-12-12T09:55:42.556 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107894.log 2023-12-12T09:55:42.556 INFO:teuthology.orchestra.run.smithi072.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102827.log.gz 2023-12-12T09:55:42.556 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104922.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104922.log.gz 2023-12-12T09:55:42.557 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95348.log 2023-12-12T09:55:42.557 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107894.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107894.log.gz 2023-12-12T09:55:42.557 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102367.log 2023-12-12T09:55:42.560 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93601.log 2023-12-12T09:55:42.560 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88649.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88649.log.gz 2023-12-12T09:55:42.560 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-12-12T09:55:42.560 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.93601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.93601.log.gz 2023-12-12T09:55:42.561 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88769.log 2023-12-12T09:55:42.561 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-12T09:55:42.561 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103041.log 2023-12-12T09:55:42.562 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95348.log.gz 2023-12-12T09:55:42.562 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102367.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102367.log.gz 2023-12-12T09:55:42.562 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104586.log 2023-12-12T09:55:42.562 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106293.log 2023-12-12T09:55:42.562 INFO:teuthology.orchestra.run.smithi072.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103041.log.gz 2023-12-12T09:55:42.562 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104586.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104586.log.gz 2023-12-12T09:55:42.562 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96827.log 2023-12-12T09:55:42.564 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88769.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.88769.log.gz 2023-12-12T09:55:42.565 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-12-12T09:55:42.565 INFO:teuthology.orchestra.run.smithi113.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-12T09:55:42.565 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.b.log: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-12-12T09:55:42.565 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102055.log 2023-12-12T09:55:42.566 INFO:teuthology.orchestra.run.smithi072.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.106293.log.gz 2023-12-12T09:55:42.566 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94068.log 2023-12-12T09:55:42.566 INFO:teuthology.orchestra.run.smithi072.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96827.log.gz 2023-12-12T09:55:42.566 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102055.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102055.log.gz 2023-12-12T09:55:42.566 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96270.log 2023-12-12T09:55:42.566 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94068.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94068.log.gz 2023-12-12T09:55:42.566 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88341.log 2023-12-12T09:55:42.567 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-12-12T09:55:42.568 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96270.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96270.log.gz 2023-12-12T09:55:42.568 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101278.log 2023-12-12T09:55:42.568 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-12-12T09:55:42.569 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.88341.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88341.log.gz 2023-12-12T09:55:42.569 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102659.log 2023-12-12T09:55:42.569 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101278.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101278.log.gz 2023-12-12T09:55:42.570 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94638.log 2023-12-12T09:55:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-12-12T09:55:42.571 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102659.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102659.log.gz 2023-12-12T09:55:42.571 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100033.log 2023-12-12T09:55:42.572 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107978.log 2023-12-12T09:55:42.572 INFO:teuthology.orchestra.run.smithi072.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.94638.log.gz 2023-12-12T09:55:42.573 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100369.log 2023-12-12T09:55:42.574 INFO:teuthology.orchestra.run.smithi072.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100033.log.gz 2023-12-12T09:55:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107978.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107978.log.gz 2023-12-12T09:55:42.575 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95432.log 2023-12-12T09:55:42.576 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100369.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100369.log.gz 2023-12-12T09:55:42.576 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101608.log 2023-12-12T09:55:42.577 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95432.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95432.log.gz 2023-12-12T09:55:42.578 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100117.log 2023-12-12T09:55:42.579 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101608.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101608.log.gz 2023-12-12T09:55:42.580 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101440.log 2023-12-12T09:55:42.581 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106209.log 2023-12-12T09:55:42.581 INFO:teuthology.orchestra.run.smithi072.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100117.log.gz 2023-12-12T09:55:42.582 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101440.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101440.log.gz 2023-12-12T09:55:42.582 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95784.log 2023-12-12T09:55:42.584 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106209.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106209.log.gz 2023-12-12T09:55:42.584 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104544.log 2023-12-12T09:55:42.584 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95784.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95784.log.gz 2023-12-12T09:55:42.585 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103293.log 2023-12-12T09:55:42.586 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104544.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104544.log.gz 2023-12-12T09:55:42.586 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107014.log 2023-12-12T09:55:42.588 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103293.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103293.log.gz 2023-12-12T09:55:42.589 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97205.log 2023-12-12T09:55:42.589 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107014.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107014.log.gz 2023-12-12T09:55:42.589 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108482.log 2023-12-12T09:55:42.591 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97205.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97205.log.gz 2023-12-12T09:55:42.591 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105258.log 2023-12-12T09:55:42.591 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107140.log 2023-12-12T09:55:42.592 INFO:teuthology.orchestra.run.smithi072.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108482.log.gz 2023-12-12T09:55:42.594 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105258.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105258.log.gz 2023-12-12T09:55:42.594 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105475.log 2023-12-12T09:55:42.594 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107140.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107140.log.gz 2023-12-12T09:55:42.594 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102913.log 2023-12-12T09:55:42.596 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103083.log 2023-12-12T09:55:42.596 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105475.log.gz 2023-12-12T09:55:42.596 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102913.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102913.log.gz 2023-12-12T09:55:42.596 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94428.log 2023-12-12T09:55:42.597 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103083.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103083.log.gz 2023-12-12T09:55:42.597 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105556.log 2023-12-12T09:55:42.598 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99571.log 2023-12-12T09:55:42.598 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94428.log.gz 2023-12-12T09:55:42.599 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105556.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105556.log.gz 2023-12-12T09:55:42.599 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98733.log 2023-12-12T09:55:42.600 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99571.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99571.log.gz 2023-12-12T09:55:42.600 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99865.log 2023-12-12T09:55:42.601 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.98733.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98733.log.gz 2023-12-12T09:55:42.602 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99739.log 2023-12-12T09:55:42.602 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99865.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99865.log.gz 2023-12-12T09:55:42.602 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106041.log 2023-12-12T09:55:42.604 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104670.log 2023-12-12T09:55:42.604 INFO:teuthology.orchestra.run.smithi072.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99739.log.gz 2023-12-12T09:55:42.605 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106041.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106041.log.gz 2023-12-12T09:55:42.605 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105090.log 2023-12-12T09:55:42.606 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106546.log 2023-12-12T09:55:42.606 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104670.log.gz 2023-12-12T09:55:42.607 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105090.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105090.log.gz 2023-12-12T09:55:42.607 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102287.log 2023-12-12T09:55:42.608 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.0.106546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106546.log.gz 2023-12-12T09:55:42.609 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106920.log 2023-12-12T09:55:42.610 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103209.log 2023-12-12T09:55:42.610 INFO:teuthology.orchestra.run.smithi072.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102287.log.gz 2023-12-12T09:55:42.611 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106920.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106920.log.gz 2023-12-12T09:55:42.611 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97161.log 2023-12-12T09:55:42.612 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103209.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103209.log.gz 2023-12-12T09:55:42.613 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96526.log 2023-12-12T09:55:42.614 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100690.log 2023-12-12T09:55:42.614 INFO:teuthology.orchestra.run.smithi072.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97161.log.gz 2023-12-12T09:55:42.615 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96526.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96526.log.gz 2023-12-12T09:55:42.615 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103125.log 2023-12-12T09:55:42.616 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100690.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100690.log.gz 2023-12-12T09:55:42.616 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102617.log 2023-12-12T09:55:42.617 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103125.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103125.log.gz 2023-12-12T09:55:42.617 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106794.log 2023-12-12T09:55:42.618 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102617.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102617.log.gz 2023-12-12T09:55:42.618 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97202.log 2023-12-12T09:55:42.619 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106794.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106794.log.gz 2023-12-12T09:55:42.620 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101398.log 2023-12-12T09:55:42.621 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99205.log 2023-12-12T09:55:42.621 INFO:teuthology.orchestra.run.smithi072.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97202.log.gz 2023-12-12T09:55:42.622 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101398.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101398.log.gz 2023-12-12T09:55:42.622 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99318.log 2023-12-12T09:55:42.623 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98118.log 2023-12-12T09:55:42.623 INFO:teuthology.orchestra.run.smithi072.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99205.log.gz 2023-12-12T09:55:42.624 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99318.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99318.log.gz 2023-12-12T09:55:42.624 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101152.log 2023-12-12T09:55:42.626 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.98118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104838.log 2023-12-12T09:55:42.626 INFO:teuthology.orchestra.run.smithi072.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.98118.log.gz 2023-12-12T09:55:42.627 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101152.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101152.log.gz 2023-12-12T09:55:42.627 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104366.log 2023-12-12T09:55:42.628 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104838.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104838.log.gz 2023-12-12T09:55:42.628 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104145.log 2023-12-12T09:55:42.629 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104366.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104366.log.gz 2023-12-12T09:55:42.630 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102099.log 2023-12-12T09:55:42.630 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104145.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104145.log.gz 2023-12-12T09:55:42.631 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97824.log 2023-12-12T09:55:42.632 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103807.log 2023-12-12T09:55:42.632 INFO:teuthology.orchestra.run.smithi072.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.102099.log.gz 2023-12-12T09:55:42.633 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97824.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97824.log.gz 2023-12-12T09:55:42.633 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105174.log 2023-12-12T09:55:42.634 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103807.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103807.log.gz 2023-12-12T09:55:42.635 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95923.log 2023-12-12T09:55:42.636 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105683.log 2023-12-12T09:55:42.636 INFO:teuthology.orchestra.run.smithi072.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105174.log.gz 2023-12-12T09:55:42.637 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95923.log.gz 2023-12-12T09:55:42.637 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96854.log 2023-12-12T09:55:42.638 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105683.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105683.log.gz 2023-12-12T09:55:42.638 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96864.log 2023-12-12T09:55:42.639 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100201.log 2023-12-12T09:55:42.640 INFO:teuthology.orchestra.run.smithi072.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96854.log.gz 2023-12-12T09:55:42.641 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96864.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96864.log.gz 2023-12-12T09:55:42.641 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95124.log 2023-12-12T09:55:42.642 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100201.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100201.log.gz 2023-12-12T09:55:42.642 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101902.log 2023-12-12T09:55:42.643 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104754.log 2023-12-12T09:55:42.644 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101902.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101902.log.gz 2023-12-12T09:55:42.645 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107182.log 2023-12-12T09:55:42.645 INFO:teuthology.orchestra.run.smithi072.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.95124.log.gz 2023-12-12T09:55:42.645 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104754.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104754.log.gz 2023-12-12T09:55:42.646 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-12T09:55:42.647 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107182.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107182.log.gz 2023-12-12T09:55:42.647 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82314.log 2023-12-12T09:55:42.648 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102575.log 2023-12-12T09:55:42.649 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.tmp-client.admin.82314.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82314.log.gz 2023-12-12T09:55:42.649 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107056.log 2023-12-12T09:55:42.649 INFO:teuthology.orchestra.run.smithi072.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-12T09:55:42.650 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102575.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102575.log.gz 2023-12-12T09:55:42.651 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102533.log 2023-12-12T09:55:42.652 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107056.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107056.log.gz 2023-12-12T09:55:42.652 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108608.log 2023-12-12T09:55:42.653 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102533.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.102533.log.gz 2023-12-12T09:55:42.653 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95616.log 2023-12-12T09:55:42.654 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108608.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108608.log.gz 2023-12-12T09:55:42.655 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103767.log 2023-12-12T09:55:42.655 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95616.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95616.log.gz 2023-12-12T09:55:42.655 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107098.log 2023-12-12T09:55:42.657 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103767.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103767.log.gz 2023-12-12T09:55:42.657 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105132.log 2023-12-12T09:55:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107098.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107098.log.gz 2023-12-12T09:55:42.658 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98034.log 2023-12-12T09:55:42.659 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105132.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105132.log.gz 2023-12-12T09:55:42.660 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101356.log 2023-12-12T09:55:42.661 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.98034.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98034.log.gz 2023-12-12T09:55:42.661 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100852.log 2023-12-12T09:55:42.662 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101356.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101356.log.gz 2023-12-12T09:55:42.662 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97176.log 2023-12-12T09:55:42.663 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100852.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100852.log.gz 2023-12-12T09:55:42.663 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97614.log 2023-12-12T09:55:42.664 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106251.log 2023-12-12T09:55:42.665 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97176.log.gz 2023-12-12T09:55:42.665 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97614.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97614.log.gz 2023-12-12T09:55:42.666 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96535.log 2023-12-12T09:55:42.667 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106251.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106251.log.gz 2023-12-12T09:55:42.667 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108794.log 2023-12-12T09:55:42.668 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100606.log 2023-12-12T09:55:42.668 INFO:teuthology.orchestra.run.smithi072.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96535.log.gz 2023-12-12T09:55:42.669 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108794.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108794.log.gz 2023-12-12T09:55:42.669 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101068.log 2023-12-12T09:55:42.670 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100606.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100606.log.gz 2023-12-12T09:55:42.671 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99487.log 2023-12-12T09:55:42.672 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101068.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.101068.log.gz 2023-12-12T09:55:42.672 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.99033.log 2023-12-12T09:55:42.673 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99487.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99487.log.gz 2023-12-12T09:55:42.673 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106878.log 2023-12-12T09:55:42.674 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.2.99033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.99033.log.gz 2023-12-12T09:55:42.674 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100564.log 2023-12-12T09:55:42.675 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106878.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.106878.log.gz 2023-12-12T09:55:42.675 INFO:teuthology.orchestra.run.smithi072.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105813.log 2023-12-12T09:55:42.677 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100564.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100564.log.gz 2023-12-12T09:55:42.677 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107434.log 2023-12-12T09:55:42.678 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105813.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105813.log.gz 2023-12-12T09:55:42.678 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108272.log 2023-12-12T09:55:42.679 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107434.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107434.log.gz 2023-12-12T09:55:42.680 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96102.log 2023-12-12T09:55:42.681 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108272.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108272.log.gz 2023-12-12T09:55:42.681 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100978.log 2023-12-12T09:55:42.682 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96102.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96102.log.gz 2023-12-12T09:55:42.683 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102743.log 2023-12-12T09:55:42.684 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100978.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100978.log.gz 2023-12-12T09:55:42.684 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105897.log 2023-12-12T09:55:42.685 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105006.log 2023-12-12T09:55:42.685 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102743.log.gz 2023-12-12T09:55:42.686 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105897.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105897.log.gz 2023-12-12T09:55:42.686 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105855.log 2023-12-12T09:55:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105006.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105006.log.gz 2023-12-12T09:55:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102160.log 2023-12-12T09:55:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105855.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105855.log.gz 2023-12-12T09:55:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104240.log 2023-12-12T09:55:42.690 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102160.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102160.log.gz 2023-12-12T09:55:42.691 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105598.log 2023-12-12T09:55:42.691 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104240.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104240.log.gzgzip 2023-12-12T09:55:42.691 INFO:teuthology.orchestra.run.smithi072.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-12-12T09:55:42.692 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105598.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105598.log.gz 2023-12-12T09:55:42.693 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94344.log 2023-12-12T09:55:42.693 INFO:teuthology.orchestra.run.smithi113.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-12-12T09:55:42.693 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108104.log 2023-12-12T09:55:42.695 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94344.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94344.log.gz 2023-12-12T09:55:42.695 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96017.log 2023-12-12T09:55:42.696 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97908.log 2023-12-12T09:55:42.696 INFO:teuthology.orchestra.run.smithi072.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108104.log.gz 2023-12-12T09:55:42.697 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96017.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96017.log.gz 2023-12-12T09:55:42.698 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106836.log 2023-12-12T09:55:42.699 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97908.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97908.log.gz 2023-12-12T09:55:42.699 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97698.log 2023-12-12T09:55:42.700 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106836.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106836.log.gz 2023-12-12T09:55:42.700 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102203.log 2023-12-12T09:55:42.701 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97698.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97698.log.gz 2023-12-12T09:55:42.701 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105641.log 2023-12-12T09:55:42.702 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102203.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.102203.log.gz 2023-12-12T09:55:42.702 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102957.log 2023-12-12T09:55:42.704 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105641.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105641.log.gz 2023-12-12T09:55:42.704 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96872.log 2023-12-12T09:55:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102957.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102957.log.gz 2023-12-12T09:55:42.705 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105300.log 2023-12-12T09:55:42.706 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96872.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96872.log.gz 2023-12-12T09:55:42.706 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103419.log 2023-12-12T09:55:42.707 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105300.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105300.log.gz 2023-12-12T09:55:42.707 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97446.log 2023-12-12T09:55:42.708 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99781.log 2023-12-12T09:55:42.709 INFO:teuthology.orchestra.run.smithi072.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.103419.log.gz 2023-12-12T09:55:42.710 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97446.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97446.log.gz 2023-12-12T09:55:42.710 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99907.log 2023-12-12T09:55:42.711 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99781.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99781.log.gz 2023-12-12T09:55:42.711 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96515.log 2023-12-12T09:55:42.712 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99907.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99907.log.gz 2023-12-12T09:55:42.712 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97488.log 2023-12-12T09:55:42.713 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96515.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.96515.log.gz 2023-12-12T09:55:42.713 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104880.log 2023-12-12T09:55:42.714 INFO:teuthology.orchestra.run.smithi113.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-12-12T09:55:42.715 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97488.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97488.log.gz 2023-12-12T09:55:42.715 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107936.log 2023-12-12T09:55:42.716 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102245.log 2023-12-12T09:55:42.716 INFO:teuthology.orchestra.run.smithi072.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104880.log.gz 2023-12-12T09:55:42.717 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107936.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107936.log.gz 2023-12-12T09:55:42.717 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107350.log 2023-12-12T09:55:42.718 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102245.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102245.log.gz 2023-12-12T09:55:42.718 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97740.log 2023-12-12T09:55:42.719 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107350.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107350.log.gz 2023-12-12T09:55:42.719 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101566.log 2023-12-12T09:55:42.720 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97740.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97740.log.gz 2023-12-12T09:55:42.721 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100894.log 2023-12-12T09:55:42.722 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101566.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101566.log.gzgzip 2023-12-12T09:55:42.722 INFO:teuthology.orchestra.run.smithi072.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97656.log 2023-12-12T09:55:42.723 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100522.log 2023-12-12T09:55:42.723 INFO:teuthology.orchestra.run.smithi072.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100894.log.gz 2023-12-12T09:55:42.724 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97656.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97656.log.gz 2023-12-12T09:55:42.724 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105048.log 2023-12-12T09:55:42.726 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100522.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100522.log.gz 2023-12-12T09:55:42.726 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103485.log 2023-12-12T09:55:42.727 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105048.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105048.log.gz 2023-12-12T09:55:42.727 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96871.log 2023-12-12T09:55:42.728 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103485.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103485.log.gz 2023-12-12T09:55:42.728 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93755.log 2023-12-12T09:55:42.729 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96871.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96871.log.gz 2023-12-12T09:55:42.729 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108440.log 2023-12-12T09:55:42.731 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.93755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93755.log.gz 2023-12-12T09:55:42.731 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99361.log 2023-12-12T09:55:42.732 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108440.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108440.log.gz 2023-12-12T09:55:42.732 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104024.log 2023-12-12T09:55:42.733 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99361.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99361.log.gz 2023-12-12T09:55:42.733 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96838.log 2023-12-12T09:55:42.734 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104024.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104024.log.gz 2023-12-12T09:55:42.734 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100327.log 2023-12-12T09:55:42.736 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96838.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96838.log.gz 2023-12-12T09:55:42.736 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107308.log 2023-12-12T09:55:42.737 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100327.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100327.log.gz 2023-12-12T09:55:42.737 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103880.log 2023-12-12T09:55:42.738 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107308.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107308.log.gz 2023-12-12T09:55:42.738 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105216.log 2023-12-12T09:55:42.739 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103880.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103880.log.gz 2023-12-12T09:55:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104408.log 2023-12-12T09:55:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105216.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105216.log.gz 2023-12-12T09:55:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108692.log 2023-12-12T09:55:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104408.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104408.log.gz 2023-12-12T09:55:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96060.log 2023-12-12T09:55:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108692.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.108692.log.gz -5 2023-12-12T09:55:42.743 INFO:teuthology.orchestra.run.smithi072.stderr: --verbose -- /var/log/ceph/ceph-osd.3.log 2023-12-12T09:55:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96060.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96060.log.gz 2023-12-12T09:55:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97530.log 2023-12-12T09:55:42.745 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107392.log 2023-12-12T09:55:42.747 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97530.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97530.log.gz 2023-12-12T09:55:42.747 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101194.log 2023-12-12T09:55:42.748 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107392.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107392.log.gz 2023-12-12T09:55:42.749 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88597.log 2023-12-12T09:55:42.750 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97116.log 2023-12-12T09:55:42.751 INFO:teuthology.orchestra.run.smithi072.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101194.log.gz 2023-12-12T09:55:42.753 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.88597.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88597.log.gz 2023-12-12T09:55:42.753 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101818.log 2023-12-12T09:55:42.754 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97116.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97116.log.gz 2023-12-12T09:55:42.754 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107476.log 2023-12-12T09:55:42.754 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108020.log 2023-12-12T09:55:42.755 INFO:teuthology.orchestra.run.smithi072.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101818.log.gz 2023-12-12T09:55:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107476.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107476.log.gz 2023-12-12T09:55:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101110.log 2023-12-12T09:55:42.757 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108020.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108020.log.gz 2023-12-12T09:55:42.758 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100936.log 2023-12-12T09:55:42.759 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102005.log 2023-12-12T09:55:42.759 INFO:teuthology.orchestra.run.smithi072.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101110.log.gz 2023-12-12T09:55:42.761 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100936.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100936.log.gz 2023-12-12T09:55:42.761 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106125.log 2023-12-12T09:55:42.762 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102005.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102005.log.gz 2023-12-12T09:55:42.762 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101776.log 2023-12-12T09:55:42.763 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95474.log 2023-12-12T09:55:42.764 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106125.log.gz 2023-12-12T09:55:42.765 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101776.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101776.log.gz 2023-12-12T09:55:42.766 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97572.log 2023-12-12T09:55:42.767 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95474.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95474.log.gz 2023-12-12T09:55:42.767 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103335.log 2023-12-12T09:55:42.768 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107644.log 2023-12-12T09:55:42.769 INFO:teuthology.orchestra.run.smithi072.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97572.log.gz 2023-12-12T09:55:42.770 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103335.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103335.log.gz 2023-12-12T09:55:42.770 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101524.log 2023-12-12T09:55:42.771 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107644.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107644.log.gz 2023-12-12T09:55:42.771 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99403.log 2023-12-12T09:55:42.772 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95742.log 2023-12-12T09:55:42.773 INFO:teuthology.orchestra.run.smithi072.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101524.log.gz 2023-12-12T09:55:42.774 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99403.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99403.log.gz 2023-12-12T09:55:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94470.log 2023-12-12T09:55:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94512.log 2023-12-12T09:55:42.776 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95742.log.gz 2023-12-12T09:55:42.777 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106167.log 2023-12-12T09:55:42.778 INFO:teuthology.orchestra.run.smithi072.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94470.log.gz 2023-12-12T09:55:42.779 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94512.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94512.log.gz 2023-12-12T09:55:42.779 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108650.log 2023-12-12T09:55:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94596.log 2023-12-12T09:55:42.781 INFO:teuthology.orchestra.run.smithi072.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106167.log.gz 2023-12-12T09:55:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108650.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108650.log.gz 2023-12-12T09:55:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105939.log 2023-12-12T09:55:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94596.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94596.log.gz 2023-12-12T09:55:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97208.log 2023-12-12T09:55:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96144.log 2023-12-12T09:55:42.785 INFO:teuthology.orchestra.run.smithi072.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105939.log.gz 2023-12-12T09:55:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97208.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97208.log.gz 2023-12-12T09:55:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107224.log 2023-12-12T09:55:42.789 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96144.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96144.log.gz 2023-12-12T09:55:42.789 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101236.log 2023-12-12T09:55:42.790 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101944.log 2023-12-12T09:55:42.790 INFO:teuthology.orchestra.run.smithi072.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107224.log.gz 2023-12-12T09:55:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101236.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101236.log.gz 2023-12-12T09:55:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-12-12T09:55:42.793 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101944.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101944.log.gz 2023-12-12T09:55:42.793 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100648.log 2023-12-12T09:55:42.794 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99163.log 2023-12-12T09:55:42.795 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100648.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100648.log.gz 2023-12-12T09:55:42.796 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103940.log 2023-12-12T09:55:42.797 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99163.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99163.log.gz 2023-12-12T09:55:42.797 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97992.log 2023-12-12T09:55:42.799 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103940.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103940.log.gz 2023-12-12T09:55:42.799 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100480.log 2023-12-12T09:55:42.802 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100159.log 2023-12-12T09:55:42.802 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97992.log.gz 2023-12-12T09:55:42.803 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88709.log 2023-12-12T09:55:42.803 INFO:teuthology.orchestra.run.smithi072.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100480.log.gz 2023-12-12T09:55:42.804 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102999.log 88.6% 2023-12-12T09:55:42.804 INFO:teuthology.orchestra.run.smithi072.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100159.log.gz 2023-12-12T09:55:42.806 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.88709.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88709.log.gz 2023-12-12T09:55:42.806 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98598.log 2023-12-12T09:55:42.807 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101650.log 2023-12-12T09:55:42.808 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102999.log.gz 2023-12-12T09:55:42.809 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.0.98598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.98598.log.gz 2023-12-12T09:55:42.809 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96530.log 2023-12-12T09:55:42.811 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101650.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101650.log.gz -5 2023-12-12T09:55:42.811 INFO:teuthology.orchestra.run.smithi072.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103982.log 2023-12-12T09:55:42.812 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96530.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.96530.log.gz 2023-12-12T09:55:42.813 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103672.log 2023-12-12T09:55:42.813 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97866.log 2023-12-12T09:55:42.814 INFO:teuthology.orchestra.run.smithi072.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103982.log.gz 2023-12-12T09:55:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.0.103672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103672.log.gz 2023-12-12T09:55:42.815 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96520.log 2023-12-12T09:55:42.816 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107518.log 2023-12-12T09:55:42.817 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97866.log.gz 2023-12-12T09:55:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96520.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96520.log.gz 2023-12-12T09:55:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93836.log 2023-12-12T09:55:42.820 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96354.log 2023-12-12T09:55:42.820 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107518.log.gz 2023-12-12T09:55:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.93836.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93836.log.gz 2023-12-12T09:55:42.822 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98076.log 2023-12-12T09:55:42.823 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96354.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96354.log.gz 2023-12-12T09:55:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108188.log 2023-12-12T09:55:42.827 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.98076.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98076.log.gz 2023-12-12T09:55:42.828 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105999.log 2023-12-12T09:55:42.828 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108188.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108188.log.gz 2023-12-12T09:55:42.828 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105725.log 2023-12-12T09:55:42.828 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105999.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105999.log.gz 2023-12-12T09:55:42.828 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108230.log 2023-12-12T09:55:42.829 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105725.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105725.log.gz 2023-12-12T09:55:42.829 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-12-12T09:55:42.830 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108230.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108230.log.gz 2023-12-12T09:55:42.831 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96834.log 2023-12-12T09:55:42.832 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101482.log 2023-12-12T09:55:42.834 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104796.log 2023-12-12T09:55:42.834 INFO:teuthology.orchestra.run.smithi072.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96834.log.gz 2023-12-12T09:55:42.835 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94386.log 2023-12-12T09:55:42.835 INFO:teuthology.orchestra.run.smithi072.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101482.log.gz 2023-12-12T09:55:42.837 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99991.log 2023-12-12T09:55:42.837 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104796.log.gz 2023-12-12T09:55:42.838 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94386.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94386.log.gz 2023-12-12T09:55:42.838 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99655.log 2023-12-12T09:55:42.840 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99991.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99991.log.gz 2023-12-12T09:55:42.840 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107602.log 2023-12-12T09:55:42.842 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99949.log 2023-12-12T09:55:42.842 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99655.log.gz 2023-12-12T09:55:42.843 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96523.log 2023-12-12T09:55:42.843 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107602.log.gz 2023-12-12T09:55:42.845 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100243.log 88.5% 2023-12-12T09:55:42.845 INFO:teuthology.orchestra.run.smithi072.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99949.log.gz 2023-12-12T09:55:42.846 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97782.log 2023-12-12T09:55:42.846 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96523.log.gz 2023-12-12T09:55:42.848 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97209.log 2023-12-12T09:55:42.848 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100243.log.gz 2023-12-12T09:55:42.849 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94206.log 2023-12-12T09:55:42.849 INFO:teuthology.orchestra.run.smithi072.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97782.log.gz 2023-12-12T09:55:42.851 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97165.log 2023-12-12T09:55:42.851 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97209.log.gz 2023-12-12T09:55:42.852 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93914.log 2023-12-12T09:55:42.852 INFO:teuthology.orchestra.run.smithi072.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94206.log.gz 2023-12-12T09:55:42.854 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97165.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97165.log.gzgzip 2023-12-12T09:55:42.854 INFO:teuthology.orchestra.run.smithi072.stderr: -5 --verbose -- /var/log/ceph/ceph.log 2023-12-12T09:55:42.855 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.93914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103251.log 2023-12-12T09:55:42.855 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93914.log.gz 2023-12-12T09:55:42.857 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94110.log 2023-12-12T09:55:42.858 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.103251.log: 85.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-12T09:55:42.858 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103251.log.gz 2023-12-12T09:55:42.859 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100768.log 2023-12-12T09:55:42.860 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100810.log 2023-12-12T09:55:42.860 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94110.log.gz 2023-12-12T09:55:42.860 INFO:teuthology.orchestra.run.smithi072.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-12-12T09:55:42.861 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100768.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100768.log.gz 2023-12-12T09:55:42.862 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99697.log 2023-12-12T09:55:42.862 INFO:teuthology.orchestra.run.smithi072.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-12-12T09:55:42.863 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100810.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100810.log.gz 2023-12-12T09:55:42.863 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104712.log 2023-12-12T09:55:42.864 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99697.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99697.log.gz 2023-12-12T09:55:42.865 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-12-12T09:55:42.866 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96186.log 2023-12-12T09:55:42.866 INFO:teuthology.orchestra.run.smithi072.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104712.log.gz 2023-12-12T09:55:42.867 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100285.log 2023-12-12T09:55:42.869 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99445.log 2023-12-12T09:55:42.869 INFO:teuthology.orchestra.run.smithi072.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96186.log.gz 2023-12-12T09:55:42.870 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95558.log 2023-12-12T09:55:42.871 INFO:teuthology.orchestra.run.smithi072.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100285.log.gz 2023-12-12T09:55:42.872 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.99445.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-12-12T09:55:42.872 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99445.log.gz 2023-12-12T09:55:42.874 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101734.log 2023-12-12T09:55:42.874 INFO:teuthology.orchestra.run.smithi072.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95558.log.gz 2023-12-12T09:55:42.875 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95390.log 2023-12-12T09:55:42.876 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101734.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.101734.log.gz -5 2023-12-12T09:55:42.876 INFO:teuthology.orchestra.run.smithi072.stderr: --verbose -- /var/log/ceph/ceph-client.admin.93969.log 2023-12-12T09:55:42.878 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106359.log 2023-12-12T09:55:42.878 INFO:teuthology.orchestra.run.smithi072.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95390.log.gz 2023-12-12T09:55:42.879 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.93969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108566.log 2023-12-12T09:55:42.880 INFO:teuthology.orchestra.run.smithi072.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93969.log.gz 2023-12-12T09:55:42.881 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106083.log 2023-12-12T09:55:42.881 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106359.log.gz 2023-12-12T09:55:42.883 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98282.log 2023-12-12T09:55:42.883 INFO:teuthology.orchestra.run.smithi072.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108566.log.gz 2023-12-12T09:55:42.884 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.106083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102869.log 2023-12-12T09:55:42.884 INFO:teuthology.orchestra.run.smithi072.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106083.log.gz 2023-12-12T09:55:42.885 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.98282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96312.log 2023-12-12T09:55:42.886 INFO:teuthology.orchestra.run.smithi072.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98282.log.gz 2023-12-12T09:55:42.887 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108524.log 2023-12-12T09:55:42.887 INFO:teuthology.orchestra.run.smithi072.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102869.log.gz 2023-12-12T09:55:42.888 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96312.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.96312.log.gz -5 2023-12-12T09:55:42.889 INFO:teuthology.orchestra.run.smithi072.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102448.log 2023-12-12T09:55:42.890 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107560.log 2023-12-12T09:55:42.890 INFO:teuthology.orchestra.run.smithi072.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108524.log.gz 2023-12-12T09:55:42.892 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96536.log 2023-12-12T09:55:42.892 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102448.log.gz 2023-12-12T09:55:42.893 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97950.log 2023-12-12T09:55:42.893 INFO:teuthology.orchestra.run.smithi072.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107560.log.gz 2023-12-12T09:55:42.895 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105769.log 2023-12-12T09:55:42.895 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96536.log.gz 2023-12-12T09:55:42.896 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.97950.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97950.log.gz 2023-12-12T09:55:42.897 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104066.log 2023-12-12T09:55:42.899 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105769.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104502.log 2023-12-12T09:55:42.899 INFO:teuthology.orchestra.run.smithi072.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105769.log.gz 2023-12-12T09:55:42.900 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93659.log 2023-12-12T09:55:42.900 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104066.log.gz 2023-12-12T09:55:42.902 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108356.log 2023-12-12T09:55:42.902 INFO:teuthology.orchestra.run.smithi072.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104502.log.gz 2023-12-12T09:55:42.903 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.93659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108062.log 2023-12-12T09:55:42.904 INFO:teuthology.orchestra.run.smithi072.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93659.log.gz 2023-12-12T09:55:42.905 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108356.log: gzip -5 --verbose 87.7% -- /var/log/ceph/ceph-client.admin.104324.log -- replaced with /var/log/ceph/ceph-client.admin.108356.log.gz 2023-12-12T09:55:42.905 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-12-12T09:55:42.906 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105395.log 2023-12-12T09:55:42.907 INFO:teuthology.orchestra.run.smithi072.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108062.log.gz 2023-12-12T09:55:42.908 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104089.log 2023-12-12T09:55:42.908 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104324.log.gz 2023-12-12T09:55:42.909 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.105395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101860.log 2023-12-12T09:55:42.910 INFO:teuthology.orchestra.run.smithi072.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105395.log.gz 2023-12-12T09:55:42.911 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94300.log 2023-12-12T09:55:42.911 INFO:teuthology.orchestra.run.smithi072.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.104089.log.gz 2023-12-12T09:55:42.912 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.101860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95969.log 2023-12-12T09:55:42.913 INFO:teuthology.orchestra.run.smithi072.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101860.log.gz 2023-12-12T09:55:42.914 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104964.log 2023-12-12T09:55:42.915 INFO:teuthology.orchestra.run.smithi072.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94300.log.gz 2023-12-12T09:55:42.915 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108314.log 2023-12-12T09:55:42.916 INFO:teuthology.orchestra.run.smithi072.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95969.log.gz 2023-12-12T09:55:42.917 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108876.log 2023-12-12T09:55:42.917 INFO:teuthology.orchestra.run.smithi072.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.104964.log.gz 2023-12-12T09:55:42.918 INFO:teuthology.orchestra.run.smithi072.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-12-12T09:55:42.919 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104282.log 2023-12-12T09:55:42.919 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108314.log.gz 2023-12-12T09:55:42.920 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108876.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108876.log.gz 2023-12-12T09:55:42.920 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108834.log 2023-12-12T09:55:42.922 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102785.log 2023-12-12T09:55:42.922 INFO:teuthology.orchestra.run.smithi072.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104282.log.gz 2023-12-12T09:55:42.924 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108146.log 2023-12-12T09:55:42.924 INFO:teuthology.orchestra.run.smithi072.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108834.log.gz 2023-12-12T09:55:42.925 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94024.log 2023-12-12T09:55:42.925 INFO:teuthology.orchestra.run.smithi072.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102785.log.gz 2023-12-12T09:55:42.926 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.108146.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.108146.log.gz 2023-12-12T09:55:42.926 INFO:teuthology.orchestra.run.smithi072.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-12-12T09:55:42.928 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95700.log 2023-12-12T09:55:42.928 INFO:teuthology.orchestra.run.smithi072.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94024.log.gz 2023-12-12T09:55:42.929 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107266.log 2023-12-12T09:55:42.930 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95700.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-12-12T09:55:42.931 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100075.log 2023-12-12T09:55:42.931 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95700.log.gz 2023-12-12T09:55:42.932 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.107266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95826.log 2023-12-12T09:55:42.932 INFO:teuthology.orchestra.run.smithi072.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107266.log.gz 2023-12-12T09:55:42.934 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.100075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102701.log 2023-12-12T09:55:42.934 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100075.log.gz 2023-12-12T09:55:42.936 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.95826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96531.log 2023-12-12T09:55:42.936 INFO:teuthology.orchestra.run.smithi072.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95826.log.gz 2023-12-12T09:55:42.937 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.102701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94704.log 2023-12-12T09:55:42.937 INFO:teuthology.orchestra.run.smithi072.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102701.log.gz 2023-12-12T09:55:42.939 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.96531.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96531.log.gz 2023-12-12T09:55:42.940 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.94704.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94704.log.gz 2023-12-12T09:55:43.531 INFO:teuthology.orchestra.run.smithi113.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-12-12T09:55:43.604 INFO:teuthology.orchestra.run.smithi113.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-12-12T09:55:43.639 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-12-12T09:55:43.782 INFO:teuthology.orchestra.run.smithi072.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-12-12T09:55:43.802 INFO:teuthology.orchestra.run.smithi113.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-12-12T09:55:43.814 INFO:teuthology.orchestra.run.smithi072.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-12-12T09:55:43.828 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-12-12T09:55:43.832 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-12-12T09:55:43.832 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m1.350s 2023-12-12T09:55:43.833 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m4.169s 2023-12-12T09:55:43.833 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m1.191s 2023-12-12T09:55:43.911 INFO:teuthology.orchestra.run.smithi113.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-12-12T09:55:44.824 INFO:teuthology.orchestra.run.smithi113.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-12-12T09:55:44.827 INFO:teuthology.orchestra.run.smithi113.stderr: 2023-12-12T09:55:44.827 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m2.333s 2023-12-12T09:55:44.827 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m6.751s 2023-12-12T09:55:44.827 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m0.626s 2023-12-12T09:55:44.828 INFO:tasks.ceph:Archiving logs... 2023-12-12T09:55:44.829 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/remote/smithi072/log 2023-12-12T09:55:44.831 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-12T09:55:45.791 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/remote/smithi113/log 2023-12-12T09:55:45.792 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-12T09:55:46.745 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-12T09:55:46.755 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-12T09:55:46.755 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-12T09:55:46.765 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-12T09:55:46.926 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-12T09:55:46.935 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-12T09:55:46.936 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T09:55:46.945 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T09:55:46.994 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2023-12-12T09:55:47.003 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2023-12-12T09:55:47.004 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T09:55:47.004 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2023-12-12T09:55:47.004 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +1884us[+1884us] +/- 84ms 2023-12-12T09:55:47.004 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -782us[ -782us] +/- 52ms 2023-12-12T09:55:47.005 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -448us[ -448us] +/- 64ms 2023-12-12T09:55:47.005 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:55:47.013 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T09:55:47.013 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2023-12-12T09:55:47.013 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +1881us[+1881us] +/- 84ms 2023-12-12T09:55:47.013 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -769us[ -768us] +/- 52ms 2023-12-12T09:55:47.013 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -411us[ -411us] +/- 64ms 2023-12-12T09:55:47.014 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:55:47.015 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-12T09:55:47.027 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-12T09:55:47.028 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-12T09:55:47.043 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-12T09:55:47.312 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-12T09:55:47.564 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T09:55:47.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:55:47.675 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T09:55:47.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:55:47.783 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-12T09:55:47.793 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-12T09:55:47.803 INFO:teuthology.task.internal:Duration was 1035.927925 seconds 2023-12-12T09:55:47.804 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-12T09:55:47.813 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-12T09:55:47.813 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-12T09:55:47.822 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-12T09:55:47.985 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T09:55:47.997 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T09:55:48.335 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-12T09:55:48.335 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2023-12-12T09:55:48.335 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-12T09:55:48.415 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2023-12-12T09:55:48.415 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-12T09:55:48.488 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-12T09:55:48.488 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-12T09:55:48.497 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-12T09:55:48.593 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-12T09:55:48.594 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-12T09:55:48.600 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-12T09:55:48.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-12T09:55:48.795 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-12T09:55:48.795 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-12T09:55:48.889 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-12T09:55:48.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-12T09:55:49.007 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-12T09:55:49.016 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-12T09:55:49.118 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2023-12-12T09:55:49.132 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2023-12-12T09:55:49.183 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-12T09:55:49.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:55:49.238 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-12T09:55:49.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:55:49.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-12T09:55:49.316 INFO:teuthology.task.internal:Transferring archived files... 2023-12-12T09:55:49.317 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/remote/smithi072 2023-12-12T09:55:49.317 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-12T09:55:49.434 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219/remote/smithi113 2023-12-12T09:55:49.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-12T09:55:49.548 INFO:teuthology.task.internal:Removing archive directory... 2023-12-12T09:55:49.548 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-12T09:55:49.558 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-12T09:55:49.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-12T09:55:49.629 INFO:teuthology.task.internal:Not uploading archives. 2023-12-12T09:55:49.629 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-12T09:55:49.639 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-12T09:55:49.639 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-12T09:55:49.648 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-12T09:55:49.700 INFO:teuthology.orchestra.run.smithi072.stdout: 393569 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 12 09:55 /home/ubuntu/cephtest 2023-12-12T09:55:49.706 INFO:teuthology.orchestra.run.smithi113.stdout: 393784 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 12 09:55 /home/ubuntu/cephtest 2023-12-12T09:55:49.710 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-12T09:55:49.721 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-12T09:55:49.742 INFO:teuthology.nuke:Checking targets against current locks 2023-12-12T09:55:49.774 DEBUG:teuthology.nuke:shortname: smithi072 2023-12-12T09:55:49.775 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-12T09:55:49.795 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:28:48.430886', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEU3ThwljGX0CbUGQkQeI+yI9VjYeXL1STRtECiXaiXc61gFHpsM3Tixc+RkxyInCiWcLjawjrMcsJUQgPtJVZU='} 2023-12-12T09:55:49.799 DEBUG:teuthology.nuke:shortname: smithi113 2023-12-12T09:55:49.800 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-12T09:55:49.822 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:28:48.432884', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6xr8M/tal4s4/lDEJJ6DdIoRSlvP3w48DiCcXXkVGWpUf2hlO+7UPcu9si7kAnZ4/agxdjzwt+2abI7ruRLJg='} 2023-12-12T09:55:49.860 INFO:teuthology.orchestra.console.smithi072:Power off 2023-12-12T09:55:49.860 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-12T09:55:49.886 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2023-12-12T09:55:49.887 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T09:55:49.906 INFO:teuthology.orchestra.console.smithi113:Power off 2023-12-12T09:55:49.906 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-12T09:55:49.921 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2023-12-12T09:55:49.934 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2023-12-12T09:55:49.934 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T09:55:49.960 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2023-12-12T09:55:53.923 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T09:55:53.961 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T09:55:54.034 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2023-12-12T09:55:54.074 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2023-12-12T09:55:58.038 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T09:55:58.075 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T09:55:58.150 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2023-12-12T09:55:58.189 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2023-12-12T09:55:58.251 INFO:teuthology.orchestra.console.smithi072:Power off completed 2023-12-12T09:56:02.194 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T09:56:02.304 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2023-12-12T09:56:02.405 INFO:teuthology.orchestra.console.smithi113:Power off completed 2023-12-12T09:56:02.510 INFO:teuthology.run:Summary data: client.0-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c client.1-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} duration: 1035.9279246330261 failure_reason: 'Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays)' flavor: default owner: mchangir sentry_event: https://sentry.ceph.com/organizations/ceph/?query=bdeee2297b444d0396a66fc8d3cf51cd status: fail success: false 2023-12-12T09:56:02.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-12T09:56:02.590 INFO:teuthology.run:FAIL